An image model optimization method and device based on a sparse matrix algorithm

Abstract

The present disclosure provides a sparse matrix algorithm-based image model optimization method and device, which comprises the following steps: inputting a pre-acquired image training set into a current to-be-optimized image model for training, and obtaining a to-be-processed disturbance parameter corresponding to the current to-be-optimized image model; obtaining a current sparse mask; based on the current sparse mask and the to-be-processed disturbance parameter, calculating a current disturbance parameter; based on the current disturbance parameter, using a gradient descent random gradient descent method to obtain updated model parameters; in the case that the image model corresponding to the updated model parameters has not converged, taking the image model corresponding to the updated model parameters as the current to-be-optimized image model, and returning to the step of inputting the pre-acquired image training set into the current to-be-optimized image model for training until the current to-be-optimized image model converges, and obtaining an optimized image model. In this way, the generalization ability of the optimized image model can be further improved, and the accuracy of image recognition by the optimized image model can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an image model optimization method and apparatus based on a sparse matrix algorithm. Background Technology

[0002] The development of deep learning has benefited from massive amounts of data. In the field of image processing, image models trained on massive amounts of image data are prone to converge to more acute local optima in the optimization space, which weakens the generalization ability of image models.

[0003] When the generalization ability of an image model weakens, the accuracy of its image recognition decreases. For example, when recognizing image data not included in the training set—that is, when the image model is recognizing previously unrecognized images—weaker generalization ability leads to lower recognition accuracy. Similarly, when the image model is recognizing low-resolution images, weaker generalization ability results in lower recognition accuracy.

[0004] Therefore, there is an urgent need for a method that can improve the generalization ability of image models, thereby improving the accuracy of image recognition. Summary of the Invention

[0005] This disclosure provides an image model optimization method and apparatus based on a sparse matrix algorithm to address the poor generalization ability of image models in the prior art, thereby further improving the generalization ability of image models.

[0006] This disclosure provides an image model optimization method based on a sparse matrix algorithm, including:

[0007] The pre-acquired image training set is input into the current image model to be optimized for training, and the perturbation parameters to be processed corresponding to the current image model to be optimized are obtained, wherein the perturbation parameters to be processed are parameters obtained based on the parameters of the current image model to be optimized;

[0008] Obtain the current sparse mask, wherein the current sparse mask is a sparse mask that represents the current optimization state of the current image model to be optimized;

[0009] Based on the current sparse mask and the perturbation parameters to be processed, the current perturbation parameters are calculated, and based on the current perturbation parameters, the stochastic gradient descent method is used to obtain the updated model parameters;

[0010] If the image model corresponding to the updated model parameters has not converged, the image model corresponding to the updated model parameters is taken as the current image model to be optimized, and the step of inputting the pre-acquired image training set into the current image model to be optimized for training is returned until the current image model to be optimized converges, and an optimized image model is obtained. The optimized image model is used for image recognition of the image to be processed.

[0011] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, before the step of obtaining the current sparse mask, the method further includes:

[0012] Determine whether the sparse mask to be determined needs to be updated, wherein the sparse mask to be determined is the sparse mask representing the previous optimization state of the current image model to be optimized.

[0013] If it is determined that the sparse mask to be determined needs to be updated, an updated sparse mask is obtained based on the sparsity, and the updated sparse mask is used as the current sparse mask, wherein the sparsity is the sparsity of the sparse mask to be determined.

[0014] If it is determined that the sparse mask to be determined does not need to be updated, the sparse mask to be determined is used as the current sparse mask.

[0015] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, the step of obtaining the perturbation parameters to be processed corresponding to the current image model to be optimized includes:

[0016] A predetermined number of image training samples are extracted from the image training set as image samples to be processed;

[0017] Calculate the first loss function corresponding to the image sample to be processed;

[0018] The gradient of the first loss function with respect to the parameters of the current image model to be optimized is used as the first gradient parameter;

[0019] Based on the first gradient parameter and the magnitude of the pre-acquired perturbation parameter, the perturbation parameter to be processed is determined.

[0020] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, the step of obtaining and updating the sparse mask based on sparsity includes:

[0021] For each image sample to be processed, obtain the Fisher information content of that image sample;

[0022] Based on the sparsity, determine the adjustment amount;

[0023] Among the multiple image samples to be processed, the image sample to be processed corresponding to the Fisher information content that satisfies the second preset condition is determined, and a corresponding number of image samples to be processed are selected according to the adjustment quantity as target image samples, and the mask of the target image sample is set to 1.

[0024] Set the mask corresponding to the remaining image samples to 0 to obtain the updated sparse mask.

[0025] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, the step of obtaining and updating the sparse mask based on sparsity includes:

[0026] Based on the preset pruning rate and the sparsity, the number of abandoned perturbations and the number of restored perturbations are obtained, wherein the number of abandoned perturbations and the number of restored perturbations are equal;

[0027] Based on the number of abandoned perturbations and the number of restored perturbations, the masks that satisfy the first preset condition in the sparse mask to be determined are set to 1, and the masks that do not satisfy the first preset condition in the sparse mask to be determined are set to 0, so as to obtain the updated sparse mask.

[0028] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, the step of setting the mask that satisfies the first preset condition in the sparse mask to be determined to 1 based on the number of abandoned perturbations and the number of restored perturbations includes:

[0029] Based on the first gradient parameter, delete the number of abandoned perturbation masks from the masks with a mask value of 1 in the sparse mask to be determined;

[0030] The remaining masks with a value of 1 in the sparse mask to be determined are determined as masks that satisfy the first preset condition, and the mask value is kept to be 1.

[0031] From the sparse masks with a mask value of 0, randomly select a number of masks to restore the perturbation, and set the randomly selected masks to satisfy the first preset condition and set them to 1.

[0032] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, the step of calculating the current perturbation parameter based on the current sparse mask and the perturbation parameter to be processed includes:

[0033] The current perturbation parameters are obtained by calculating the Hadam product of the current sparse mask and the perturbation parameters to be processed.

[0034] According to the image model optimization method based on sparse matrix algorithm provided in this disclosure, the step of obtaining updated model parameters based on the current perturbation parameters using stochastic gradient descent includes:

[0035] The sum of the current perturbation parameters and the parameters of the current image model to be optimized is used as the parameters to be processed;

[0036] Calculate the second loss function corresponding to the parameters to be processed, and use the gradient of the second loss function with respect to the parameters of the current image model to be optimized as the second gradient parameter;

[0037] The product of the pre-acquired learning rate and the second gradient parameter is used as the third gradient parameter;

[0038] The difference between the parameters of the current image model to be optimized and the third gradient parameters is used as the updated model parameters.

[0039] This disclosure also provides an image model optimization device based on a sparse matrix algorithm, the device comprising:

[0040] The first acquisition module is used to input a pre-acquired image training set into the current image model to be optimized for training, and to acquire the perturbation parameters to be processed corresponding to the current image model to be optimized, wherein the perturbation parameters to be processed are parameters obtained based on the parameters of the current image model to be optimized;

[0041] The second acquisition module is used to acquire the current sparse mask, wherein the current sparse mask is a sparse mask that represents the current optimization state of the current image model to be optimized.

[0042] The calculation module is used to calculate the current perturbation parameters based on the current sparse mask and the perturbation parameters to be processed, and to obtain the updated model parameters based on the current perturbation parameters using the stochastic gradient descent method.

[0043] The training module is used to, when the image model corresponding to the updated model parameters has not converged, take the image model corresponding to the updated model parameters as the current image model to be optimized, and return to the step of inputting the pre-acquired image training set into the current image model to be optimized for training, until the current image model to be optimized converges to obtain an optimized image model, wherein the optimized image model is used for image recognition of the image to be processed.

[0044] This disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of an image model optimization method based on a sparse matrix algorithm as described in any of the preceding claims.

[0045] This disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of an image model optimization method based on a sparse matrix algorithm as described in any of the preceding claims.

[0046] This disclosure provides an image model optimization method and apparatus based on a sparse matrix algorithm. It utilizes a current sparse mask to sparsify the perturbation parameters, obtaining the current perturbation parameters. Then, based on these current perturbation parameters, it employs gradient descent and stochastic gradient descent to obtain updated model parameters, resulting in an optimized image model. The current sparse mask represents the current optimization state of the image model to be optimized. During image model optimization, the importance of the parameters of the current image model to be optimized can be quantified based on the current sparse mask corresponding to the current optimization state, thereby determining whether to perturb those parameters. This further improves the generalization ability of the optimized image model, and subsequently enhances the accuracy of image recognition of the processed image using the optimized image model. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is one of the flowcharts illustrating an image model optimization method based on a sparse matrix algorithm provided in this disclosure;

[0049] Figure 2 This is the second flowchart of an image model optimization method based on a sparse matrix algorithm provided in this disclosure;

[0050] Figure 3 This is the third flowchart of an image model optimization method based on a sparse matrix algorithm provided in this disclosure;

[0051] Figure 4 This is the fourth flowchart of an image model optimization method based on a sparse matrix algorithm provided in this disclosure;

[0052] Figure 5 This is a schematic diagram of the structure of an image model optimization device based on a sparse matrix algorithm provided in this disclosure;

[0053] Figure 6 This is a schematic diagram of the structure of the electronic device provided in this disclosure. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this disclosure.

[0055] To further improve the generalization ability of the model, this disclosure provides an image model optimization method, apparatus, electronic device, non-transitory computer-readable storage medium, and computer program product based on a sparse matrix algorithm. The following is in conjunction with… Figure 1 This disclosure describes an image model optimization method based on a sparse matrix algorithm according to an embodiment of the present disclosure.

[0056] like Figure 1 As shown, this disclosure provides an image model optimization method based on a sparse matrix algorithm, including:

[0057] S101, input the pre-acquired image training set into the current image model to be optimized for training, and obtain the perturbation parameters to be processed corresponding to the current image model to be optimized.

[0058] When optimizing an image model, an image training set can be acquired and then input into the current image model to be optimized for training. The current image model to be optimized can be an optimized version of the pre-acquired image model; in other words, it can be a parameter-adjusted image model. When the pre-acquired image training set is input into the current image model for the first time, the current image model to be optimized is the pre-acquired image model.

[0059] After training the current image model by inputting the image training set, the optimization result output by the current image model can be obtained. Based on the optimization result, the perturbation parameters to be processed, i.e., the perturbation vector to be processed, can be obtained for the current image model. The perturbation parameters to be processed are obtained based on the parameters of the current image model.

[0060] In one implementation, a loss function can be calculated based on the optimization results, the gradient of the loss function with respect to the parameters of the current image model to be optimized can be obtained, and then the perturbation parameters to be processed can be determined based on the gradient of the loss function with respect to the parameters of the current image model to be optimized and the magnitude of the pre-acquired perturbation parameters.

[0061] In one implementation, the image training set includes multiple image training samples. The loss function corresponding to all image training samples included in the image training set can be calculated, and then the gradient of the loss function with respect to the parameters of the current image model to be optimized can be obtained, thereby determining the perturbation parameters to be processed.

[0062] S102, obtain the current sparse mask.

[0063] After obtaining the perturbation parameters to be processed, a current sparse mask can be obtained to sparsify these parameters. The current sparse mask represents the current optimization state of the image model to be optimized. It is a binary mask containing only 0 and 1 mask values ​​(elemental values). The number of mask elements in the current sparse mask is consistent with the number of parameters in the current image model to be optimized, meaning they have the same dimensionality.

[0064] In one implementation, the Fisher information masking method or the dynamic sparse masking method can be used to obtain the current sparse mask; both are reasonable. The Fisher information masking method provides higher accuracy, while the dynamic sparse masking method is faster. The choice can be made based on actual needs. Of course, other sparse masking search algorithms can also be used to obtain the current sparse mask; no specific limitation is made here.

[0065] As one implementation method, when the pre-acquired image training set is input into the current image model to be optimized for the first time, a pre-set sparse mask can be used as the current sparse mask, or the Fisher information mask method can be used to obtain the current sparse mask. Both are reasonable and will not be specifically limited here.

[0066] S103. Based on the current sparse mask and the perturbation parameters to be processed, calculate the current perturbation parameters, and based on the current perturbation parameters, use the stochastic gradient descent method to obtain the updated model parameters.

[0067] After obtaining the perturbation parameters to be processed and the current sparse mask, the current perturbation parameters can be calculated based on the current sparse mask and the perturbation parameters to be processed. The current sparse mask includes mask values ​​of 0 and 1. Therefore, the current perturbation parameters obtained based on the current sparse mask and the perturbation parameters to be processed have many parameters that are covered by 0 compared to the perturbation parameters to be processed, thus achieving sparsification of the perturbation parameters to be processed.

[0068] In other words, the embodiments of this disclosure can quantify the importance of the parameters of the current image model to be optimized based on the current sparse mask corresponding to the current optimization state of the current image model to be optimized, thereby determining whether to perturb the parameters so as to improve the generalization ability of the image model in the future.

[0069] Since the current sparse mask is a sparse mask that can characterize the current optimization state of the current image model to be optimized, the process of obtaining the current perturbation parameters based on the current sparse mask and the perturbation parameters to be processed is equivalent to the process of quantifying the importance of the parameters of the current image model to be optimized based on the current optimization state of the current image model to be optimized.

[0070] Then, based on the current perturbation parameters, stochastic gradient descent can be used to obtain the updated model parameters. After obtaining the updated model parameters, it can be determined whether the image model corresponding to the updated model parameters has converged.

[0071] The embodiments of this disclosure use stochastic gradient descent to obtain updated model parameters. Compared with current sharpness perception algorithms, the embodiments of this disclosure do not require redefining the quantization algorithm, thus reducing the amount of computation.

[0072] S104, if the image model corresponding to the updated model parameters has not converged, the image model corresponding to the updated model parameters is taken as the current image model to be optimized, and the step of inputting the pre-acquired image training set into the current image model to be optimized for training is returned until the current image model to be optimized converges, and the optimized image model is obtained.

[0073] If the image model corresponding to the updated model parameters does not converge, the image model corresponding to the updated model parameters can be used as the current image model to be optimized. The step of inputting the pre-acquired image training set into the current image model to be optimized for training is returned, thereby iteratively optimizing the image model. During the entire optimization process, the parameters corresponding to the mask value of 0 are not calculated. Compared with the current sharpness perception algorithm, the amount of computation can be greatly reduced, thereby speeding up the optimization of the image model to be optimized, until the current image model to be optimized converges, and the optimized image model is obtained. The optimized image model is used for image recognition of the image to be processed.

[0074] In other words, if the image model corresponding to the updated model parameters converges, the image model corresponding to the updated model parameters can be taken as the optimized image model. This completes the optimization of the image model to be optimized.

[0075] After obtaining the optimized image model, it can be used to perform image recognition on the image to be processed, thereby obtaining the image recognition result. Because the optimized image model has a higher generalization ability than the current image model, it can more accurately recognize images (images to be processed) that the current image model cannot accurately identify. For example, the image to be processed may be an image that the current model has not processed, or it may be a low-resolution image.

[0076] In other words, using an optimized image model to perform image recognition on the image to be processed can further improve the accuracy of image recognition compared to the current image model-based image recognition method.

[0077] As can be seen, the embodiments of this disclosure can use the current sparse mask to sparsify the perturbation parameters to be processed, obtaining the current perturbation parameters. Then, based on the current perturbation parameters, the updated model parameters can be obtained using gradient descent and stochastic gradient descent methods, thus obtaining an optimized image model. Here, the current sparse mask represents the current optimization state of the image model to be optimized. During the optimization process of the image model, the importance of the parameters of the current image model to be optimized can be quantified based on the current sparse mask corresponding to the current optimization state, thereby determining whether to perturb the parameter. This further improves the generalization ability of the optimized image model. Furthermore, sparsifying the perturbation parameters to be processed and using stochastic gradient descent methods can reduce the computational load and improve the optimization speed of the image model. In the subsequent image recognition process using the optimized image model, the accuracy of image recognition of the processed image can be further improved.

[0078] This disclosure provides an image model optimization method based on a sparse matrix algorithm, which is an end-to-end method for finding the optimal perturbation parameters. Compared with the global perturbation method of the current sharpness perception algorithm, it can not only further improve the generalization ability of the optimized image model, but also reduce the amount of computation and improve the optimization speed of the image model.

[0079] As one embodiment of this disclosure, before the step of obtaining the current sparse mask described above, the method may further include:

[0080] Determine whether the sparse mask to be determined needs to be updated, wherein the sparse mask to be determined is the sparse mask representing the previous optimization state of the current image model to be optimized.

[0081] During the iterative optimization of the image model to be optimized, the changes in the image model parameters are small between adjacent preset number of iterations. Therefore, within the preset number of iterations, the same sparse mask can be used to represent the optimization state corresponding to the preset number of iterations.

[0082] To further reduce computation, the mask update frequency can be preset, that is, the sparse mask is updated once every time the image training set is input into the current image model to be optimized for training reaches a preset number of times.

[0083] The preset number of iterations can be set according to actual needs. For example, the preset number of iterations can be 4, 10, or 32. Correspondingly, the mask update frequency can be updated once every 4 training iterations of the current image training set input into the current image to be optimized. Alternatively, the mask update frequency can be updated once every 10 training iterations of the current image training set input into the current image to be optimized. Or, the mask update frequency can be updated once every 32 training iterations of the current image training set input into the current image to be optimized. These are all reasonable options and are not specifically limited here.

[0084] Before obtaining the current sparse mask, it can be determined whether the sparse mask to be determined needs to be updated, which means determining whether the number of training iterations of the image training set input into the current image model to be optimized is a multiple of the preset number of iterations. Here, the sparse mask to be determined is the sparse mask representing the previous optimization state of the current image model to be optimized.

[0085] In one implementation, it can be determined whether there is a remainder when the number of times the image training set is input into the current image model to be optimized is divided by a preset number of times. If there is a remainder, it means that the sparse mask to be determined does not need to be updated. If there is no remainder, it means that the sparse mask to be determined needs to be updated.

[0086] If it is determined that the sparse mask to be determined needs to be updated, the updated sparse mask can be obtained based on the sparsity, and the updated sparse mask can be used as the current sparse mask. Here, the sparsity is the sparsity of the sparse mask to be determined, which can characterize the number of masks with a mask value of 1 included in the sparse mask to be determined.

[0087] In one implementation, the updated sparse mask can be obtained based on sparsity using the Fisher information masking method, or it can be obtained using a dynamic sparse masking method. Both approaches are reasonable.

[0088] As one implementation method, during the iterative optimization of the image model to be optimized, the Fisher information mask method can always be used to obtain and update the sparse mask. Alternatively, the dynamic sparse mask method can always be used to obtain and update the sparse mask.

[0089] If it is determined that the sparse mask to be determined does not need to be updated, it can be used as the current sparse mask. Therefore, this embodiment can determine whether the sparse mask to be determined needs updating, and thus obtain the current sparse mask according to different situations. Since it is not necessary to re-obtain the sparse mask for each iteration of the image model being optimized, the computational load can be further reduced, thereby improving the optimization speed of the model. Furthermore, during the iterative optimization of the image model, the current sparse mask can be optimized simultaneously, instead of using random sampling, which can further improve the model's generalization ability.

[0090] As one implementation of this disclosure, such as Figure 2 As shown, the steps described above for obtaining the perturbation parameters corresponding to the current image model to be optimized may include:

[0091] S201, a preset number of image training samples are extracted from the image training set as image samples to be processed.

[0092] In the iterative optimization process of the image model to be optimized, each time the current image training set is input into the current image model to be optimized for training, all image training samples included in the image training set are input into the current image model to be optimized for training.

[0093] To further reduce computational load, a predetermined number of image training samples can be extracted from the image training set as the images to be processed. The predetermined number can be set based on actual conditions; for example, it could be 100, 500, or 688. These are all reasonable options and are not specifically limited here. In other words, the images to be processed include multiple samples.

[0094] S202, calculate the first loss function corresponding to the image sample to be processed.

[0095] After obtaining the image sample to be processed, the first loss function of the current image model to be optimized corresponding to the image sample to be processed can be calculated. That is, the loss function corresponding to the training of the current image model by inputting the image sample to be processed is the first loss function.

[0096] In one implementation, the empirical loss function of the current image model to be optimized corresponding to the image sample to be processed can be calculated as the first loss function, wherein the formula of the empirical loss function is as shown in (1):

[0097]

[0098] Where x is the image sample to be processed, x iLet i be the image sample to be processed with index i, i.e. the i-th image sample to be processed, N be the number of image samples to be processed, w be the parameters of the current image model to be optimized, and f(w; x) be the first loss function, which can also be expressed as f(w).

[0099] S203, the gradient of the first loss function with respect to the parameters of the current image model to be optimized is used as the first gradient parameter.

[0100] After obtaining the first loss function, the gradient of the first loss function with respect to the parameters of the current image model to be optimized can be used to obtain the first gradient parameters, which can also be called the first vector. The first gradient parameters can be obtained using... express.

[0101] S204, Based on the first gradient parameter and the magnitude of the pre-acquired perturbation parameter, determine the perturbation parameter to be processed.

[0102] After obtaining the first gradient parameters, the perturbation parameters to be processed can be determined based on the first gradient parameters and the amplitude of the pre-obtained perturbation parameters. In one embodiment, the calculation formula for the perturbation parameters to be processed can be as shown in (2):

[0103]

[0104] Where ∈1 represents the perturbation parameter to be processed, and ρ represents the amplitude of the perturbation parameter, which is the amplitude of the perturbation parameter to be processed. for The 2-norm.

[0105] As can be seen, the embodiments of this disclosure can extract a preset number of image training samples from the image training set as image samples to be processed, and then calculate a first loss function based on the image samples to be processed, thereby determining the perturbation parameters to be processed. In this way, the amount of computation can be further reduced, thereby improving the optimization speed of the image model.

[0106] As one implementation of this disclosure, such as Figure 3 As shown, the steps for obtaining and updating the sparse mask based on sparsity can include:

[0107] S301, for each of the image samples to be processed, obtain the Fisher information of the image sample to be processed.

[0108] For each image sample to be processed, the Fisher information content corresponding to each image sample can be calculated. In one embodiment, the empirical Fisher information content corresponding to each image sample to be processed can be calculated. The formula for calculating the empirical Fisher information content corresponding to the i-th image sample to be processed can be as shown in (3):

[0109]

[0110] Among them, F wi Let N be the empirical Fisher information content (Fischer information content) corresponding to the i-th image sample to be processed. F p represents the number of Fisher information samples, which is the number of image samples to be processed. w (y i |x i ) is given x i Below, the predicted distribution of the current image model to be optimized with parameter w.

[0111] S302, Based on the sparsity, determine the adjustment quantity.

[0112] To ensure that the sparsity of the current sparse mask is consistent during the iterative optimization of the image model to be optimized, the adjustment quantity can be determined based on the sparsity when obtaining the updated sparse mask. The adjustment quantity is the number of masks with a mask value of 1 included in the sparse mask to be determined.

[0113] S303, among the multiple image samples to be processed, determine the image sample to be processed corresponding to the Fisher information content that satisfies the second preset condition, and select a corresponding number of image samples to be processed as target image samples according to the adjustment quantity, and set the mask of the target image sample to 1.

[0114] After obtaining the Fisher information content and adjustment quantity for each image sample to be processed, the image sample to be processed corresponding to the Fisher information content that meets the second preset condition can be determined, and the corresponding number of image samples to be processed can be selected as target image samples according to the adjustment quantity, and the mask of the target image sample can be set to 1.

[0115] In one implementation, the second preset condition can be a preset Fisher information content threshold. In this way, it is possible to determine the image samples to be processed that have a Fisher information content greater than the Fisher information content threshold, and then randomly select an adjusted number of image samples to be processed as target image samples, and set the mask of the target image samples to 1.

[0116] In another implementation, the second preset condition can be a preset rank position after sorting the Fisher information content of each image sample to be processed. For example, the second preset condition can be that the Fisher information content of each image sample to be processed is sorted in descending order and ranks in the top 30. In this way, the image samples to be processed corresponding to the Fisher information content and ranked in the top 30 can be determined. Then, a preset number of image samples to be processed are randomly selected from the image samples ranked in the top 30 as target image samples, and the mask of the target image samples is set to 1.

[0117] In one implementation, the Fisher information content of each image sample to be processed can be sorted in descending order, and the top-ranked image samples of a certain number can be selected as target image samples, with their masks set to 1. In other words, a preset number of image samples with the largest Fisher information content are selected as target image samples, and their masks are set to 1.

[0118] S304, set the mask corresponding to the remaining image samples to 0 to obtain the updated sparse mask.

[0119] After setting the target image sample value to 1, the masks corresponding to the remaining image samples to be processed can be set to 0. Then, the masks with a value of 1 and the masks with a value of 0 can be combined into a matrix to obtain the updated sparse mask.

[0120] Steps S301-S304 above describe the method of obtaining the updated sparse mask using the Fisher information masking method, which is essentially the method of obtaining the current sparse mask using the Fisher information masking method. The Fisher information masking method obtains the current narrative mask with higher accuracy, which can further improve the generalization ability of the model.

[0121] As one implementation of this disclosure, such as Figure 4 As shown, the steps for obtaining and updating the sparse mask based on sparsity can include:

[0122] S401, based on the preset pruning rate and the sparsity, obtain the number of abandoned perturbations and the number of restored perturbations.

[0123] To obtain an updated sparse mask, the number of discarded perturbations and the number of restored perturbations can be obtained based on a preset pruning rate and sparsity. In one implementation, the pruning rate can be a sample discard rate, which can be a discard rate obtained based on the image samples to be processed and the image training set.

[0124] The preset pruning rate can be a ratio set in advance according to the actual situation. The number of discarded perturbations is the number of masks with a value of 1 in the existing sparse mask (the sparse mask to be processed) that are set to 0, based on the existing sparse mask. The number of restored perturbations is the number of masks with a value of 0 in the existing sparse mask (the sparse mask to be processed) that are set to 1, based on the existing sparse mask.

[0125] To ensure that the sparsity of the current sparse matrix does not change during the iterative optimization of the image model, the number of discarded perturbations can be equal to the number of restored perturbations, that is, the number of prunings is equal to the number of restorations.

[0126] In the initial stage of iterative optimization of the image model to be optimized, a sparse mask with large variations can be obtained to optimize the image model, that is, to try more combinations of sparse masks. Therefore, a larger pruning rate can be adopted, that is, a larger number of discarded perturbations and a larger number of restored perturbations, so as to obtain the current sparse mask in the future.

[0127] Towards the end of the iterative optimization of the image model, when the current image model is close to convergence, using a large pruning rate can lead to instability in the iterative optimization of the image model. Therefore, a smaller pruning rate, that is, a smaller number of discarded perturbations and a smaller number of restored perturbations, can be used to obtain the current sparse mask in the future.

[0128] To meet the different requirements at different stages of iterative optimization of the image model to be optimized, the number of discarded perturbations and the number of restored perturbations can be obtained by using a cosine decay algorithm based on a preset pruning rate and sparsity.

[0129] The formula for the cosine decay algorithm is shown in (4):

[0130]

[0131] The formulas for calculating the number of abandoned disturbances and the number of restored disturbances can be shown in (5):

[0132]

[0133] Where, N grow To restore the number of disturbances, N drop denoted as the number of discarded perturbations, α as the pruning rate, T as the total number of optimizations, t as the current number of optimizations, s as the sparsity, and |w| as the number of parameters.

[0134] S402, based on the number of abandoned perturbations and the number of restored perturbations, set the masks that meet the first preset condition in the sparse mask to be determined to 1, and set the masks that do not meet the first preset condition in the sparse mask to be determined to 0, to obtain the updated sparse mask.

[0135] After obtaining the number of abandoned perturbations and the number of restored perturbations, the masks that meet the first preset condition in the sparse mask to be determined can be set to 1, and the masks that do not meet the first preset condition in the sparse mask to be determined can be set to 0, so as to obtain the updated sparse mask.

[0136] Steps S401-S402 above describe the method of obtaining and updating the sparse mask using dynamic sparse masking, which is essentially the method of obtaining the current sparse mask using dynamic sparse masking. Dynamic sparse masking is faster at obtaining the current sparse mask, further improving the optimization speed of the model.

[0137] As one embodiment of this disclosure, the step of setting the mask that satisfies the first preset condition in the sparse mask to be determined based on the number of abandoned perturbations and the number of restored perturbations may include:

[0138] Based on the first gradient parameter, the number of abandoned perturbation masks are deleted from the masks with a mask value of 1 in the sparse mask to be determined.

[0139] The remaining masks with a value of 1 in the sparse masks to be determined are determined as masks that satisfy the first preset condition, and the mask value is kept at 1.

[0140] To obtain the updated sparse mask, based on the first gradient parameter, a certain number of masks with discarded perturbations can be removed from the sparse mask with a mask value of 1 in the undetermined sparse mask. In other words, a certain number of masks with discarded perturbations can be selected from the sparse mask with a mask value of 1 in the undetermined sparse mask and set to 0. The remaining masks with a mask value of 1 in the undetermined sparse mask are then determined as masks that satisfy the first preset condition, and their mask values ​​are kept at 1.

[0141] In one implementation, the absolute values ​​of the first gradient parameters can be obtained, and the negative values ​​of the absolute values ​​of each first gradient parameter can be sorted in ascending order. Then, the mask with the highest number of abandoned perturbations is selected and set to 0. That is, the mask with the smallest absolute value of the first gradient parameter among the abandoned perturbation parameters is selected and set to 0.

[0142] From the sparse masks to be determined that have a mask value of 0, randomly select a number of masks to restore the perturbation. Then, determine the randomly selected masks that satisfy the second preset condition and set them to 1. In this way, the masks in the sparse masks to be determined that satisfy the first preset condition can be set to 1, achieving dynamic permutation and combination of masks for subsequent updates to the sparse mask.

[0143] As one implementation of this disclosure, the step of calculating the current perturbation parameter based on the current sparse mask and the perturbation parameter to be processed may include:

[0144] The current perturbation parameters are obtained by calculating the Hadam product of the current sparse mask and the perturbation parameters to be processed. The Hadam product can be represented as the element-wise multiplication of two matrices (vectors) of the same size.

[0145] In other words, the current sparse mask and the perturbation parameters to be processed are matrices of the same size, and the Hadam product can be obtained by multiplying the mask (elements) in the current sparse mask with the parameters (elements) in the perturbation parameters to be processed.

[0146] The formula for calculating the current disturbance parameters can be shown in (6):

[0147] ∈=∈1⊙m (6)

[0148] Where ∈ represents the current perturbation parameter, ∈1 represents the perturbation parameter to be processed, and m represents the current sparse mask.

[0149] The formula for calculating the current disturbance parameters can also be shown in (7):

[0150]

[0151] As can be seen, the embodiments of this disclosure can obtain the current perturbation parameter by calculating the Hadam product of the current sparse mask and the perturbation parameter to be processed. Since the current sparse mask can characterize the current optimization state of the image model to be optimized, and the current sparse mask can quantify the importance of the parameters of the image model to be optimized, the importance of the parameters of the image model to be optimized can be quantified based on the current optimization state of the image model to be optimized, thereby determining whether to perturb the parameter. This can further improve the generalization ability of the optimized image model.

[0152] As one embodiment of this disclosure, the step of obtaining updated model parameters based on the current perturbation parameters using stochastic gradient descent may include:

[0153] The sum of the current perturbation parameters and the parameters of the current image model to be optimized is used as the parameters to be processed.

[0154] Calculate the second loss function corresponding to the parameters to be processed, and use the gradient of the second loss function with respect to the parameters of the current image model to be optimized as the second gradient parameter.

[0155] The product of the pre-acquired learning rate and the second gradient parameter is used as the third gradient parameter.

[0156] The difference between the parameters of the current image model to be optimized and the third gradient parameters is used as the updated model parameters.

[0157] The formula for calculating the updated model parameters can be shown in (8):

[0158]

[0159]

[0160] Among them, w t Let be the parameters of the current image model to be optimized, given that the model has been iterated and optimized t times. Here, η represents the corresponding parameter to be processed, and w represents the learning rate of the current image model to be optimized. t+1 To update the corresponding model parameters, For the second loss function, is the second gradient parameter.

[0161] The formula for calculating the updated model parameters can also be shown in (9):

[0162]

[0163] Where f(w+∈) is the second loss function, The gradient of the function f(w+∈) with respect to the parameters of the current image model to be optimized is called the second gradient parameter.

[0164] As one embodiment of this disclosure, the following describes an image model optimization method based on a sparse matrix algorithm provided by this disclosure, with specific examples. The method employs the Fisher information masking method to obtain the current sparse mask, with a mask update frequency of 2:

[0165] Obtain the image training set and the image model to be optimized, model 1. At this point, image model 1 is the current image model to be optimized, and its parameters are parameter 1. Input the image training set into image model 1 for the first time for iterative optimization. Extract x image training samples from the image training set as image samples to be processed. Calculate the first loss function 1 corresponding to the image samples to be processed. Use the gradient of the first loss function with respect to the parameters of the current image model to be optimized as the first gradient parameter 1. Based on the first gradient parameter 1 and the magnitude of the pre-obtained perturbation parameters, determine the perturbation parameter 1 to be processed.

[0166] The Fisher information mask method is used to obtain the current sparse mask 1. The Hadam product of the current sparse mask 1 and the perturbation parameter 1 to be processed is calculated to obtain the current perturbation parameter 1. Based on the current perturbation parameter 1, the stochastic gradient descent method is used to obtain the updated model parameter 1.

[0167] If the image model corresponding to updated model parameter 1 fails to converge, the image model corresponding to updated model parameter 1 is used as the current image model to be optimized. The image training set is then input into the image model corresponding to updated model parameter 1 for iterative optimization. x image training samples are extracted from the image training set as image samples to be processed. The first loss function 2 corresponding to the image samples to be processed is calculated. The gradient of the first loss function with respect to the parameters of the current image model to be optimized is used as the first gradient parameter 3. Based on the first gradient parameter 1 and the magnitude of the pre-acquired perturbation parameter, the perturbation parameter 2 to be processed is determined.

[0168] If it is determined that the current sparse mask 1 does not need to be updated, that is, the current sparse mask 1 is the current sparse mask, the Hadam product of the current sparse mask 1 and the perturbation parameter 2 to be processed is calculated to obtain the current perturbation parameter 2. Based on the current perturbation parameter 2, the stochastic gradient descent method is used to obtain the updated model parameter 2.

[0169] If the image model corresponding to updated model parameter 2 does not converge, the image model corresponding to updated model parameter 2 is used as the current image model to be optimized. The image training set is input to the image model corresponding to updated model parameter 2 for the third time for iterative optimization. x image training samples are extracted from the image training set as image samples to be processed. The first loss function 3 corresponding to the image samples to be processed is calculated. The gradient of the first loss function with respect to the parameters of the current image model to be optimized is used as the first gradient parameter 3. Based on the first gradient parameter 1 and the magnitude of the pre-acquired perturbation parameter, the perturbation parameter 3 to be processed is determined.

[0170] If it is determined that the current sparse mask 1 needs to be updated, the Fisher information mask method is used to obtain the updated sparse mask, that is, the current sparse mask 2 is obtained. The current sparse mask 2 is the current sparse mask. The Hadam product of the current sparse mask 2 and the perturbation parameter 3 to be processed is calculated to obtain the current perturbation parameter 3. Based on the current perturbation parameter 3, the stochastic gradient descent method is used to obtain the updated model parameter 3.

[0171] This process continues until the updated model parameter n is obtained. Once the image model corresponding to the updated model parameter n converges, the optimized image model can be obtained. After obtaining the optimized image model, the image to be processed can be input into the optimized image model to obtain the recognition result output by the optimized image model.

[0172] As can be seen, the embodiments of this disclosure can use the current sparse mask to sparsify the perturbation parameters to be processed, obtaining the current perturbation parameters. Then, based on the current perturbation parameters, the updated model parameters can be obtained using gradient descent and stochastic gradient descent methods, thus obtaining an optimized image model. Here, the current sparse mask represents the current optimization state of the image model to be optimized. During the optimization process of the image model, the importance of the parameters of the current image model to be optimized can be quantified based on the current sparse mask corresponding to the current optimization state, thereby determining whether to perturb the parameter. This further improves the generalization ability of the optimized image model. Furthermore, sparsifying the perturbation parameters to be processed and using stochastic gradient descent methods can reduce the computational load and improve the optimization speed of the image model. In the subsequent image recognition process using the optimized image model, the accuracy of image recognition of the processed image can be further improved.

[0173] As one implementation of this disclosure, the process of rewriting the above-mentioned experience Fisher information is as follows:

[0174] Fisher's information content is defined by the formula shown in (10):

[0175]

[0176] Among them, E x～p(x) Let E be the mathematical expectation of x and the original data distribution p(x). y～p(y|x) Let p be the mathematical expectation of y and p(y|x).

[0177] In overparameterized networks, calculating the accurate Fisher information matrix is ​​tricky because the Fisher information is the square of the model parameters, i.e., |w|×|w|. Therefore, the Fisher information can be approximated as a diagonal matrix. For the calculation of the two mathematical expectations, since the original data distribution p(x) is usually unavailable, an approximation of p(x) can be taken, and the approximate formula is shown in (11):

[0178]

[0179] Regarding the approximate formula shown in (11), since the accurate labeling of the data is known in supervised learning, the mathematical expectation does not need to be calculated. Thus, the empirical Fisher information quantity can be obtained, i.e., the formula shown in (3).

[0180] As one embodiment of this disclosure, the maximum-minimum optimization problem can be expressed using the formula shown in (12) below:

[0181]

[0182] Among them, st1 T m represents the current sparse mask m under the condition of sparsity s and iterative optimization t times.

[0183] As one implementation of this disclosure, the convergence of the optimized image model obtained by the above-described image model optimization method based on a sparse matrix algorithm can be verified in the following manner.

[0184] Under the assumptions of continuity, bounded variance, and bounded gradient, the learning rate is assumed to be... And the learning rate is The rate of decay, i.e. The amplitude ρ of the disturbance parameter is expressed as √ The rate of decay, i.e.

[0185] Where L is the parameter of the continuity assumption, η t Let η0 be the learning rate for the number of iterations t, and ρ be the learning rate for the image model to be optimized. t ρ0 represents the magnitude of the perturbation parameter corresponding to the number of iterations t, and ρ0 represents the magnitude of the perturbation parameter corresponding to the image model to be optimized.

[0186] Under the condition that ρ0≤Gη0, the following convergence conclusion holds for the current sharpness sensing algorithm, as shown in Equation (13), where G is a constant with bounded gradient in the continuity assumption:

[0187]

[0188] in, w0 represents the parameters of the image model to be optimized. T The parameters of the optimized image model are σ, which is a constant representing the bounded variance under the continuity assumption.

[0189] Under the condition that ρ0≤Gη0, for the embodiments of this disclosure, the following convergence conclusion exists, as shown in formula (14):

[0190]

[0191] in,

[0192]

[0193] It can be seen that the embodiments of this disclosure have the same upper bound on the convergence rate as the current sharpness perception algorithm. In other words, the image model optimization method based on sparse matrix algorithm provided by the embodiments of this disclosure, during the iterative optimization of the image model to be optimized, not only has a convergence time no greater than that of the current sharpness perception algorithm, but can also converge to a minimum point with better generalization performance. That is, it can further improve the generalization ability of the optimized image model, so that in the subsequent image recognition process of the image to be processed through the optimized image model, the accuracy of image recognition of the image to be processed can be further improved.

[0194] The following describes an image model optimization device based on a sparse matrix algorithm provided in the embodiments of this disclosure. The image model optimization device based on a sparse matrix algorithm described below and the image model optimization method based on a sparse matrix algorithm described above can be referred to and correspond to each other.

[0195] like Figure 5 As shown, an image model optimization device based on a sparse matrix algorithm is provided. The device includes:

[0196] The first acquisition module 510 is used to input the pre-acquired image training set into the current image model to be optimized for training, and to acquire the perturbation parameters to be processed corresponding to the current image model to be optimized.

[0197] The disturbance parameters to be processed are parameters obtained based on the parameters of the current image model to be optimized.

[0198] The second acquisition module 520 is used to acquire the current sparse mask.

[0199] Wherein, the current sparse mask is a sparse mask that represents the current optimization state of the current image model to be optimized.

[0200] The calculation module 530 is used to calculate the current perturbation parameters based on the current sparse mask and the perturbation parameters to be processed, and to obtain updated model parameters based on the current perturbation parameters by using the gradient descent and stochastic gradient descent method.

[0201] The training module 540 is used to, when the image model corresponding to the updated model parameters has not converged, take the image model corresponding to the updated model parameters as the current image model to be optimized, and return to the step of inputting the pre-acquired image training set into the current image model to be optimized for training, until the current image model to be optimized converges and an optimized image model is obtained.

[0202] The optimized image model is used for image recognition of the image to be processed.

[0203] As can be seen, the image model optimization device based on a sparse matrix algorithm provided in this embodiment can use a current sparse mask to sparsify the perturbation parameters to be processed, obtaining the current perturbation parameters. Then, based on the current perturbation parameters, a gradient descent or stochastic gradient descent method can be used to obtain updated model parameters, thereby obtaining an optimized image model. Here, the current sparse mask represents the current optimization state of the image model to be optimized. During the optimization process, the importance of the parameters of the current image model to be optimized can be quantified based on the current sparse mask corresponding to the current optimization state, thereby determining whether to perturb the parameter. This further improves the generalization ability of the optimized image model. Furthermore, sparsifying the perturbation parameters and using stochastic gradient descent reduces computation and increases the optimization speed of the image model. In the subsequent image recognition process using the optimized image model, the accuracy of image recognition of the processed image can be further improved.

[0204] As one embodiment of this disclosure, the above-described apparatus may further include:

[0205] The judgment module is used to determine whether the sparse mask to be determined needs to be updated before obtaining the current sparse mask.

[0206] Wherein, the sparse mask to be determined is the sparse mask that represents the previous optimization state of the current image model to be optimized.

[0207] An update module is used to obtain an updated sparse mask based on sparsity when it is determined that the sparse mask to be determined needs to be updated, and to use the updated sparse mask as the current sparse mask.

[0208] Wherein, the sparsity is the sparsity of the sparse mask to be determined;

[0209] The determination module is used to use the sparse mask to be determined as the current sparse mask if it is determined that the sparse mask to be determined does not need to be updated.

[0210] As one embodiment of this disclosure, the first acquisition module 510 described above may include:

[0211] The extraction unit is used to extract a preset number of image training samples from the image training set as image samples to be processed.

[0212] The first computing unit is used to calculate the first loss function corresponding to the image sample to be processed.

[0213] The first gradient unit is used to take the gradient of the first loss function with respect to the parameters of the current image model to be optimized as the first gradient parameter.

[0214] The first determining unit is used to determine the perturbation parameter to be processed based on the first gradient parameter loss function and the magnitude of the pre-acquired perturbation parameter.

[0215] As one embodiment of this disclosure, the above-mentioned updating module may include:

[0216] The first acquisition unit is used to acquire the Fisher information of each image sample to be processed.

[0217] The second determining unit is used to determine the adjustment quantity based on the sparsity.

[0218] The third determining unit is used to determine, among the multiple image samples to be processed, the image sample corresponding to the Fisher information content that satisfies the second preset condition, and select a corresponding number of image samples to be processed as target image samples according to the adjustment quantity, and set the mask of the target image sample to 1.

[0219] The fourth determining unit is used to set the mask corresponding to the remaining image samples to be processed to 0, so as to obtain the updated sparse mask.

[0220] As one embodiment of this disclosure, the above-mentioned updating module may include:

[0221] The second acquisition unit is used to acquire the number of abandoned perturbations and the number of restored perturbations based on the preset pruning rate and the sparsity.

[0222] The number of abandoned disturbances is equal to the number of restoration disturbances.

[0223] The fifth determining unit is used to set the masks that meet the first preset condition in the sparse mask to be determined to 1 and the masks that do not meet the first preset condition to 0 based on the number of abandoned perturbations and the number of restored perturbations, so as to obtain the updated sparse mask.

[0224] As one embodiment of this disclosure, the fifth determining unit described above may include:

[0225] The deletion sub-unit is used to delete the number of abandoned perturbation masks from the masks with a mask value of 1 in the sparse mask to be determined, based on the first gradient parameter of the parameters of the current image model to be optimized according to the loss function.

[0226] The holding subunit is used to determine the remaining masks with a mask value of 1 in the sparse mask to be determined as masks that satisfy the first preset condition, and to keep the mask value of 1.

[0227] A sub-unit is selected to randomly select a number of masks from the masks with a mask value of 0 in the sparse mask to be determined, and then determine the randomly selected mask as a mask that satisfies the first preset condition and set it to 1.

[0228] As one embodiment of this disclosure, the computing module 530 may include:

[0229] The second calculation unit is used to calculate the Hadam product of the current sparse mask and the perturbation parameter to be processed, so as to obtain the current perturbation parameter.

[0230] As one embodiment of this disclosure, the computing module 530 may include:

[0231] The third calculation unit is used to take the sum of the current perturbation parameters and the parameters of the current image model to be optimized as the parameters to be processed.

[0232] The fourth calculation unit is used to calculate the second loss function to be processed corresponding to the parameters to be processed, and to use the gradient of the second loss function to be processed with respect to the parameters of the current image model to be optimized as the second gradient parameter to be processed.

[0233] The second gradient unit is used to take the product of the pre-acquired learning rate and the second gradient parameter to be processed as the first and third gradient parameters.

[0234] The fifth calculation unit is used to use the difference between the parameters of the current image model to be optimized and the first and third gradient parameters as the updated model parameters.

[0235] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logic instructions in the memory 630 to execute an image model optimization method based on a sparse matrix algorithm. The method includes: inputting a pre-acquired image training set into the current image model to be optimized for training, and obtaining the perturbation parameters to be processed corresponding to the current image model to be optimized, wherein the perturbation parameters to be processed are parameters obtained based on the parameters of the current image model to be optimized; obtaining the current sparse mask, wherein the current sparse mask is a sparse mask representing the current optimization state of the current image model to be optimized; calculating the current perturbation parameters based on the current sparse mask and the perturbation parameters to be processed; and obtaining the updated model parameters based on the current perturbation parameters using a stochastic gradient descent method; if the image model corresponding to the updated model parameters has not converged, taking the image model corresponding to the updated model parameters as the current image model to be optimized, and returning to the step of inputting the pre-acquired image training set into the current image model to be optimized for training, until the current image model to be optimized converges, thus obtaining an optimized image model.

[0236] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this disclosure, essentially, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0237] On the other hand, this disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute an image model optimization method based on a sparse matrix algorithm provided by the above methods.

[0238] In another aspect, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image model optimization method based on the sparse matrix algorithm provided above.

[0239] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0240] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0241] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. An image model optimization method based on a sparse matrix algorithm, characterized in that, The method includes: The pre-acquired image training set is input into the current image model to be optimized for training, and the perturbation parameters to be processed corresponding to the current image model to be optimized are obtained, wherein the perturbation parameters to be processed are parameters obtained based on the parameters of the current image model to be optimized; The method further includes: obtaining a current sparse mask, wherein the current sparse mask is a sparse mask representing the current optimization state of the current image model to be optimized; prior to obtaining the current sparse mask, the method further includes: determining whether the sparse mask to be determined needs to be updated, wherein the sparse mask to be determined is a sparse mask representing the previous optimization state of the current image model to be optimized; if it is determined that the sparse mask to be determined needs to be updated, obtaining an updated sparse mask based on sparsity, and using the updated sparse mask as the current sparse mask, wherein the sparsity is the sparsity of the sparse mask to be determined; if it is determined that the sparse mask to be determined does not need to be updated, using the sparse mask to be determined as the current sparse mask. Based on the current sparse mask and the perturbation parameters to be processed, the current perturbation parameters are calculated, and based on the current perturbation parameters, the stochastic gradient descent method is used to obtain the updated model parameters; If the image model corresponding to the updated model parameters has not converged, the image model corresponding to the updated model parameters is taken as the current image model to be optimized, and the step of inputting the pre-acquired image training set into the current image model to be optimized for training is returned until the current image model to be optimized converges, and an optimized image model is obtained. The optimized image model is used for image recognition of the image to be processed.

2. The image model optimization method based on sparse matrix algorithm according to claim 1, characterized in that, The step of obtaining the perturbation parameters to be processed corresponding to the current image model to be optimized includes: A predetermined number of image training samples are extracted from the image training set as image samples to be processed; Calculate the first loss function corresponding to the image sample to be processed; The gradient of the first loss function with respect to the parameters of the current image model to be optimized is used as the first gradient parameter; Based on the first gradient parameter and the magnitude of the pre-acquired perturbation parameter, the perturbation parameter to be processed is determined.

3. The image model optimization method based on sparse matrix algorithm according to claim 2, characterized in that, The step of obtaining and updating the sparse mask based on sparsity includes: For each image sample to be processed, obtain the Fisher information content of that image sample; Based on the sparsity, determine the adjustment amount; Among the multiple image samples to be processed, the image sample to be processed corresponding to the Fisher information content that satisfies the second preset condition is determined, and a corresponding number of image samples to be processed are selected according to the adjustment quantity as target image samples, and the mask of the target image sample is set to 1. Set the mask corresponding to the remaining image samples to 0 to obtain the updated sparse mask.

4. The image model optimization method based on sparse matrix algorithm according to claim 2, characterized in that, The step of obtaining and updating the sparse mask based on sparsity includes: Based on the preset pruning rate and the sparsity, the number of abandoned perturbations and the number of restored perturbations are obtained, wherein the number of abandoned perturbations and the number of restored perturbations are equal; Based on the number of abandoned perturbations and the number of restored perturbations, the masks that satisfy the first preset condition in the sparse mask to be determined are set to 1, and the masks that do not satisfy the first preset condition in the sparse mask to be determined are set to 0, so as to obtain the updated sparse mask.

5. The image model optimization method based on sparse matrix algorithm according to claim 4, characterized in that, The step of setting the mask that satisfies the first preset condition in the sparse mask to be determined based on the number of abandoned perturbations and the number of restored perturbations includes: Based on the first gradient parameter, delete the number of abandoned perturbation masks from the masks with a mask value of 1 in the sparse mask to be determined; The remaining masks with a value of 1 in the sparse mask to be determined are determined as masks that satisfy the first preset condition, and the mask value is kept to be 1. From the sparse masks with a mask value of 0, randomly select a number of masks to restore the perturbation, and set the randomly selected masks to satisfy the first preset condition and set them to 1.

6. The image model optimization method based on a sparse matrix algorithm according to any one of claims 1-5, characterized in that, The step of calculating the current perturbation parameter based on the current sparse mask and the perturbation parameter to be processed includes: The current perturbation parameters are obtained by calculating the Hadam product of the current sparse mask and the perturbation parameters to be processed.

7. The image model optimization method based on a sparse matrix algorithm according to any one of claims 1-5, characterized in that, The step of obtaining updated model parameters based on the current perturbation parameters using stochastic gradient descent includes: The sum of the current perturbation parameters and the parameters of the current image model to be optimized is used as the parameters to be processed; Calculate the second loss function corresponding to the parameters to be processed, and use the gradient of the second loss function with respect to the parameters of the current image model to be optimized as the second gradient parameter; The product of the pre-acquired learning rate and the second gradient parameter is used as the third gradient parameter; The difference between the parameters of the current image model to be optimized and the third gradient parameters is used as the updated model parameters.

8. An image model optimization device based on a sparse matrix algorithm, characterized in that, The device includes: The first acquisition module is used to input a pre-acquired image training set into the current image model to be optimized for training, and to acquire the perturbation parameters to be processed corresponding to the current image model to be optimized, wherein the perturbation parameters to be processed are parameters obtained based on the parameters of the current image model to be optimized; The second acquisition module is used to acquire the current sparse mask, wherein the current sparse mask is a sparse mask representing the current optimization state of the current image model to be optimized; before the step of acquiring the current sparse mask, the module further includes: determining whether the sparse mask to be determined needs to be updated, wherein the sparse mask to be determined is a sparse mask representing the previous optimization state of the current image model to be optimized; if it is determined that the sparse mask to be determined needs to be updated, acquiring an updated sparse mask based on sparsity, and using the updated sparse mask as the current sparse mask, wherein the sparsity is the sparsity of the sparse mask to be determined; if it is determined that the sparse mask to be determined does not need to be updated, using the sparse mask to be determined as the current sparse mask; The calculation module is used to calculate the current perturbation parameters based on the current sparse mask and the perturbation parameters to be processed, and to obtain the updated model parameters based on the current perturbation parameters using the stochastic gradient descent method. The training module is used to, when the image model corresponding to the updated model parameters has not converged, take the image model corresponding to the updated model parameters as the current image model to be optimized, and return to the step of inputting the pre-acquired image training set into the current image model to be optimized for training, until the current image model to be optimized converges to obtain an optimized image model, wherein the optimized image model is used for image recognition of the image to be processed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image model optimization method based on sparse matrix algorithm as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of an image model optimization method based on a sparse matrix algorithm as described in any one of claims 1 to 7.

Images