Channel coefficient adjustment method and device, storage medium and computer device

Abstract

The application discloses a channel coefficient adjustment method and device, a storage medium and a computer device. The method comprises the following steps: determining a pre-task of a data processing model based on a processing type of a multimedia task; performing data expansion processing on a first multimedia data set by using the pre-task to obtain a second multimedia data set; inputting the first multimedia data set into the data processing model to obtain a first loss parameter of the data processing model; inputting the first multimedia data set and the second multimedia data set into the data processing model for training to obtain a second loss parameter of the data processing model; and updating a channel importance coefficient of a convolution channel in the data processing model based on the first loss parameter and the second loss parameter to obtain a trained data processing model. The application improves the accuracy of data output by the data processing model.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and more specifically, to a channel coefficient adjustment method, apparatus, storage medium, and computer device. Background Technology

[0002] Machine learning is the core and foundation of artificial intelligence, and artificial neural network algorithms are a common implementation method for artificial intelligence. Specifically, it involves designing a neural network model based on artificial neural network algorithms. This neural network model includes a large number of convolutional kernels and convolutional channels. Then, the model parameters in the original neural network model are trained using a large amount of training data to obtain a trained neural network model. The trained neural network model then performs convolution operations on arbitrary input data to obtain an output data. Summary of the Invention

[0003] This application provides a channel coefficient adjustment method, apparatus, storage medium, and computer equipment, which can solve the technical problem of how to improve the accuracy of data output data from a data processing model.

[0004] In a first aspect, embodiments of this application provide a channel coefficient adjustment method, the method comprising:

[0005] The prerequisite tasks for the data processing model are determined based on the processing type of the multimedia task.

[0006] The first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset.

[0007] The first multimedia dataset is input into the data processing model to obtain the first loss parameter of the data processing model;

[0008] The first multimedia dataset and the second multimedia dataset are input into the data processing model for training to obtain the second loss parameter of the data processing model;

[0009] Based on the first loss parameter and the second loss parameter, the channel importance coefficients of the convolutional channels in the data processing model are updated to obtain the trained data processing model.

[0010] Secondly, embodiments of this application provide a channel coefficient adjustment device, comprising:

[0011] The task determination module is used to determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task.

[0012] The data processing module is used to perform data expansion processing on the first multimedia dataset using the aforementioned pre-task to obtain the second multimedia dataset.

[0013] The first acquisition module is used to input the first multimedia dataset into the data processing model to obtain the first loss parameter of the data processing model;

[0014] The second acquisition module is used to input the first multimedia dataset and the second multimedia dataset into the data processing model for training, so as to obtain the second loss parameter of the data processing model;

[0015] The coefficient update module is used to update the channel importance coefficients of the convolutional channels in the data processing model based on the first loss parameter and the second loss parameter, so as to obtain the trained data processing model.

[0016] Thirdly, embodiments of this application provide a storage medium storing a computer program adapted to be loaded by a processor and to execute the steps of the above-described method.

[0017] Fourthly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described above.

[0018] In this embodiment, based on the processing type of the multimedia task, a pre-task for the data processing model is added to expand the data of the first multimedia dataset, thereby obtaining a large amount of second multimedia dataset. Then, the first loss parameter corresponding to the first multimedia dataset is obtained, and the second loss parameter corresponding to the first and second multimedia datasets is also obtained. The channel coefficients of the convolutional channels in the data processing model are updated based on the loss parameters obtained from these two synchronous trainings. This increases the amount of training data for the data processing model, thereby improving the pruning accuracy of the data processing model. Furthermore, by adding a new path for obtaining the second loss parameter, the robustness and generalization of the data processing model are improved. In turn, by improving the pruning accuracy, robustness, and generalization of the data processing model for the processing type of multimedia tasks, the accuracy of the output data obtained by the data processing model under different scenarios or tasks is improved. At the same time, the accuracy loss of the pruning network of the data processing model during the pruning process is also reduced. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic flowchart illustrating a channel coefficient adjustment method provided in an embodiment of this application;

[0021] Figure 2 A schematic flowchart illustrating a channel coefficient adjustment method provided in an embodiment of this application;

[0022] Figure 3 A schematic flowchart illustrating a channel coefficient adjustment method provided in an embodiment of this application;

[0023] Figure 4 A schematic flowchart illustrating a channel coefficient adjustment method provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of a channel coefficient adjustment device provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the structure of a channel coefficient adjustment device provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] To make the features and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the process of inputting data into a neural network model and then performing convolution operations on the input data to obtain output data, the neural network model is computationally intensive and time-consuming due to the large number of convolution kernels and convolution channels. To reduce the computational load and thus the processing time, existing technologies employ channel pruning to simplify the neural network model, thereby improving its processing speed by reducing computational complexity.

[0029] Existing channel pruning methods train a neural network model using a set of data, and then update the channel importance coefficients of the convolutional channels in the neural network model based on the loss parameters obtained from this set of training data. Convolutional channels with channel importance coefficients less than a set threshold are the channels pruned from the neural network model. Therefore, this channel pruning method has poor robustness and generalization, which leads to poor accuracy of the output data when the neural network model obtains output data for input data in other scenarios.

[0030] The following will combine Figures 1-4 The channel coefficient adjustment method provided in the embodiments of this application will be described in detail.

[0031] Please see Figure 1 The diagram below illustrates a method for adjusting channel coefficients according to an embodiment of this application. Figure 1 As shown, the method may include the following steps S101-S105.

[0032] S101, determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task.

[0033] In one implementation, a multimedia task refers to the multimedia task performed by the data processing model, such as image classification, image segmentation, object detection, human pose detection, and speech recognition. It can be understood that image classification, image segmentation, object detection, human pose detection, and speech recognition are different processing types. The data processing model is a neural network model obtained by training an initial neural network model based on the initial training dataset corresponding to the multimedia task. It has multiple convolutional kernels and convolutional channels. Essentially, the data processing model is a pre-trained neural network model that has not undergone pruning; that is, it can generate output data that satisfies the corresponding multimedia task based on the input data, but its data processing time is relatively long.

[0034] The channel coefficient adjustment device first determines the processing type of the multimedia task to be performed by the data processing model, and then generates a pretext task for the data processing model for that processing type.

[0035] S102, the first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset.

[0036] In one implementation, a pretext task is used to generate pseudo-labeled data from a large amount of unlabeled data. The first multimedia dataset includes multiple first multimedia data sets, which can be video data, image data, audio data, etc. Different multimedia task types correspond to different data expansion methods. For example, if the multimedia task type is image data processing, the data expansion method can be image rotation, image cropping, image coloring, image flipping, etc. The data expansion method is used to obtain more multimedia data and form a pair of positive and negative samples with the original multimedia data. The second multimedia dataset includes multiple second multimedia data sets, where each first multimedia data set has a corresponding second multimedia data set. That is, after performing data expansion processing on any first multimedia data set, a second multimedia data set is obtained. The currently selected first multimedia data set and the currently generated second multimedia data set have a corresponding relationship. It can be understood that the corresponding first and second multimedia data sets have the same data source.

[0037] The channel coefficient adjustment device acquires the first multimedia dataset, and then performs data expansion processing on multiple first multimedia data in the first multimedia dataset through the pre-task corresponding to the data processing model to obtain multiple second multimedia data, i.e., the second multimedia dataset.

[0038] For example, suppose the first multimedia dataset includes four images: image A, image B, image C and image D. By performing data expansion processing on the four images in the first multimedia dataset through a pre-task, we obtain image A', image B', image C' and image D', which are four corresponding multiple second multimedia datasets.

[0039] S103, the first multimedia dataset is input into the data processing model to obtain the first loss parameter of the data processing model.

[0040] In one implementation, the channel coefficient adjustment device inputs a first multimedia dataset into a data processing model, causing the data processing model to acquire multiple first data features based on the first multimedia dataset, and then generate a first loss parameter based on the multiple first data features and the loss function in the data processing model.

[0041] S104, the first multimedia dataset and the second multimedia dataset are input into the data processing model for training to obtain the second loss parameter of the data processing model.

[0042] In one implementation, the channel coefficient adjustment device inputs a first multimedia dataset and a second multimedia dataset into a data processing model, so that the data processing model obtains multiple second data features based on the first multimedia dataset and the second multimedia dataset, and then generates a second loss parameter based on the multiple second data features and the loss function in the data processing model.

[0043] S105, based on the first loss parameter and the second loss parameter, update the channel importance coefficients of the convolutional channels in the data processing model to obtain the trained data processing model.

[0044] In one implementation, the channel coefficient adjustment device sequentially acquires the channel importance coefficients of the target convolutional channel from multiple convolutional channels of the data processing model. Then, based on the currently acquired channel importance coefficients, a set update function, a first loss parameter, and a second loss parameter, it calculates the updated channel importance coefficients and replaces the target convolutional channel's channel importance coefficients with the updated ones. The target convolutional channel is any one of the multiple convolutional channels in the data processing model. After the channel coefficient adjustment device has updated the channel importance coefficients of each convolutional channel, the trained data processing model is obtained.

[0045] It should be noted that convolutional channels with channel importance coefficients less than or equal to a set coefficient threshold are pruned channels. After receiving the input data, the data processing model only performs convolution operations on the input data through convolutional channels with channel importance coefficients greater than the set coefficient threshold, and then obtains and outputs the output data.

[0046] In one implementation, after updating the channel importance coefficients of each convolutional channel, the channel coefficient adjustment device compares the channel importance coefficients of each convolutional channel with a set coefficient threshold, and then removes the convolutional channels whose channel importance coefficients are less than or equal to the set coefficient threshold to obtain the trained data processing model.

[0047] In one implementation, a first multimedia dataset includes a large amount of first multimedia data, and a corresponding second multimedia dataset also includes a large amount of second multimedia data. The channel coefficient adjustment device first acquires a first predetermined quantity of first multimedia data from the first multimedia dataset, and simultaneously acquires a second predetermined quantity of second multimedia data from the second multimedia dataset. Then, the predetermined quantity of first multimedia data is input into the data processing model as a set of input data, allowing the model to obtain a first loss parameter based on this set of input data. Simultaneously, the predetermined quantity of first multimedia data and the predetermined quantity of second multimedia data are also input into the data processing model as a set of input data, allowing the model to obtain a second loss parameter based on this set of input data. Then, the channel importance coefficients of the convolutional channels in the data processing model are updated according to the currently obtained first and second loss parameters. Next, the device acquires another first predetermined quantity of first multimedia data from the first multimedia dataset, and simultaneously acquires a second predetermined quantity of second multimedia data from the second multimedia dataset, repeating the aforementioned process of updating the channel importance coefficients of the convolutional channels until all multimedia data in the first and second multimedia datasets has been traversed. The final data processing model obtained is the trained data processing model. It should be noted that the first multimedia dataset and the second multimedia dataset are traversed and finished simultaneously. For example, if the total number of first multimedia data in the first multimedia dataset is x, and the total number of second multimedia data in the second multimedia dataset is 2x, then the first set quantity is y, and the second set quantity is 2y, where x and y are arbitrary integers. It should be emphasized that the above is only an example for illustrative purposes to facilitate understanding of this embodiment and does not impose any specific limitations on this embodiment.

[0048] In this embodiment, based on the processing type of the multimedia task, a pre-task for the data processing model is added to expand the data of the first multimedia dataset, thereby obtaining a large amount of second multimedia dataset. Then, the first loss parameter corresponding to the first multimedia dataset is obtained, and the second loss parameter corresponding to the first and second multimedia datasets is also obtained. The channel coefficients of the convolutional channels in the data processing model are updated based on the loss parameters obtained from these two synchronous trainings. This increases the amount of training data for the data processing model, thereby improving the pruning accuracy of the data processing model. Furthermore, by adding a new path for obtaining the second loss parameter, the robustness and generalization of the data processing model are improved. In turn, by improving the pruning accuracy, robustness, and generalization of the data processing model for the processing type of multimedia tasks, the accuracy of the output data obtained by the data processing model under different scenarios or tasks is improved. At the same time, the accuracy loss of the pruning network of the data processing model during the pruning process is also reduced.

[0049] Please see Figure 2 The diagram below illustrates a method for adjusting channel coefficients according to an embodiment of this application. Figure 2 As shown, the method may include the following steps S201-S215.

[0050] S201, Obtain initial data processing network.

[0051] In one implementation, the data processing model includes a base network and a pruned network. The base network is the initial convolutional network of the data processing model, and the pruned network is a convolutional network obtained by adding channel importance coefficients to each convolutional channel of the base network. The base network includes multiple convolutional kernels and convolutional channels, and similarly, the pruned network also includes multiple convolutional kernels and convolutional channels, with the number of convolutional kernels and convolutional channels in the pruned network being the same as those in the base network.

[0052] The initial data processing network is a convolutional neural network that has been trained on the training dataset. This initial data processing network has multiple convolutional kernels and convolutional channels. For example, when the initial data processing network receives input data, it performs convolution operations on the received input data through multiple convolutional kernels and convolutional channels to obtain the convolution result, and outputs the convolution result as the data processing result for the input data.

[0053] The channel coefficient adjustment device acquires the initial data processing network that needs pruning. This can be understood as follows: the channel coefficient adjustment device can directly receive the initial data processing network sent to it by other devices; or it can acquire a training dataset, then train the original data processing network based on the training dataset to obtain a trained data processing network, and use this trained data processing network as the initial data processing network that needs pruning. It should be noted that the original data processing network refers to the original form of the convolutional neural network, i.e., an untrained convolutional neural network.

[0054] S202, set initial channel importance coefficients in multiple convolutional channels of the initial data processing network to obtain the pruned data processing network.

[0055] In one implementation, the initial channel importance coefficients are set values, which are the original values ​​used for pruning training. The pruning data processing network is the data processing network to be pruned.

[0056] The channel coefficient adjustment device replicates the initial data processing network to obtain a replicated initial data processing network. Then, it sets initial channel importance coefficients for multiple convolutional channels of the replicated initial data processing network to obtain a pruned data processing network.

[0057] S203, the initial data processing network is used as the base network of the data processing model, and the pruned data processing network is used as the pruned network of the data processing model.

[0058] In one implementation, the channel coefficient adjustment device uses the initial data processing network as the base network of the data processing model and the pruned data processing network as the pruned network of the data processing model.

[0059] By setting up a base network and a pruning network, when pruning the convolutional channels in the pruning network, the error between the convolutional results of the pruning network and the base network can be determined based on the convolutional results of the base network. This allows for the determination of the pruning method, i.e., the adjustment method for the channel importance coefficients of each convolutional channel. Thus, while ensuring the accuracy of the output data of the pruning network, the pruning network is also compressed, thereby improving the computational speed of the pruning network. In other words, compared with the base network, the output results of the pruning network are equally accurate, and the network structure of the pruning network is smaller and the computation speed is faster.

[0060] S204, Determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task.

[0061] In one implementation, a multimedia task refers to the multimedia task performed by the data processing model, such as image classification, image segmentation, object detection, human pose detection, and speech recognition. It can be understood that image classification, image segmentation, object detection, human pose detection, and speech recognition are different processing types. The data processing model is a neural network model obtained by training an initial neural network model based on the initial training dataset corresponding to the multimedia task. It has multiple convolutional kernels and convolutional channels. Essentially, the data processing model is a pre-trained neural network model that has not undergone pruning; that is, it can generate output data that satisfies the corresponding multimedia task based on the input data, but its data processing time is relatively long.

[0062] The channel coefficient adjustment device first determines the processing type of the multimedia task to be performed by the data processing model, and then generates a pretext task for the data processing model for that processing type.

[0063] S205, the first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset.

[0064] In one implementation, a pretext task is used to generate pseudo-labeled data from a large amount of unlabeled data. The first multimedia dataset includes multiple first multimedia data sets, which can be video data, image data, audio data, etc. Different multimedia task types correspond to different data expansion methods. For example, if the multimedia task type is image data processing, the data expansion method can be image rotation, image cropping, image coloring, image flipping, etc. The data expansion method is used to obtain more multimedia data and form a pair of positive and negative samples with the original multimedia data. The second multimedia dataset includes multiple second multimedia data sets, where each first multimedia data set has a corresponding second multimedia data set. That is, after performing data expansion processing on any first multimedia data set, a second multimedia data set is obtained. The currently selected first multimedia data set and the currently generated second multimedia data set have a corresponding relationship. It can be understood that the corresponding first and second multimedia data sets have the same data source.

[0065] The channel coefficient adjustment device acquires the first multimedia dataset, and then performs data expansion processing on multiple first multimedia data in the first multimedia dataset through the pre-task corresponding to the data processing model to obtain multiple second multimedia data, i.e., the second multimedia dataset.

[0066] For example, suppose the first multimedia dataset includes four images: image A, image B, image C and image D. By performing data expansion processing on the four images in the first multimedia dataset through a pre-task, we obtain image A', image B', image C' and image D', which are four corresponding multiple second multimedia datasets.

[0067] It should be noted that the pseudo-label set for this prerequisite task is... Where p is the multimedia data index of multimedia data 1, and q is the multimedia data index of multimedia data 2. When multimedia data 1 and multimedia data 2 originate from the same multimedia data, When multimedia data 1 and multimedia data 2 originate from different multimedia data sources, Multimedia data 1 and multimedia data 2 can both be any multimedia data from the first multimedia dataset and the second multimedia dataset. For example, pseudo-labels for image A and image A'. =1, pseudo-tags for images A and B =0.

[0068] S206, based on the basic network in the data processing model, perform convolution operations on multiple first multimedia data in the first multimedia dataset to obtain multiple first data features.

[0069] In one implementation, the channel coefficient adjustment device inputs the first multimedia dataset into the base network of the data processing model. The base network of the data processing model sequentially performs convolution operations on multiple first multimedia data in the first multimedia dataset to obtain multiple first data features. It can be understood that one first multimedia data corresponds to one first data feature.

[0070] S207, Based on the pruning network in the data processing model, perform convolution operations on multiple first multimedia data in the first multimedia dataset to obtain multiple second data features.

[0071] In one implementation, the channel coefficient adjustment device inputs the first multimedia dataset into the pruning network in the data processing model. The pruning network in the data processing model sequentially performs convolution operations on multiple first multimedia data in the first multimedia dataset to obtain multiple second data features. It can be understood that one first multimedia data corresponds to one second data feature.

[0072] S208, obtain the first loss parameter of the data processing model based on the plurality of first data features and the plurality of second data features.

[0073] In one implementation, the channel adjustment device uses the loss function of the data processing model to calculate the first loss parameter based on multiple first data features and multiple second data features.

[0074] For example, the loss function can be represented by the following formula:

[0075]

[0076] in, It is any first data feature; It is any second data feature; The pseudo-labels are based on the pre-set tasks. Specifically, they are the pseudo-labels corresponding to the first multimedia data corresponding to the first data feature and the first multimedia data corresponding to the second data feature; n is the pre-set edge distance parameter; B is the total number of data in the first multimedia dataset; and L is the first loss parameter.

[0077] For example, suppose the first multimedia dataset contains four images A, B, C, and D. Then, through the base network of the data processing model, the first data features A1, B1, C1, and D1 corresponding to the four images are obtained sequentially. Simultaneously, through the pruning network of the data processing model, the second data features A2, B2, C2, and D2 corresponding to the four images are obtained sequentially. Then, a first data feature A1 is obtained from the multiple first data features, and a second data feature A2 is obtained from the multiple second data features. Finally, the loss function described above is applied... Calculate the loss parameter L1 once, then obtain new first data feature B1 and new second data feature B2, calculate the new loss parameter L2, and repeat the above steps until multiple first data features and multiple second data features have been calculated. Then, sum the results based on each sub-loss parameter to obtain the first loss parameter L.

[0078] By performing convolution operations on multiple first multimedia data in the first multimedia dataset, and then obtaining the first loss parameter based on the convolution operation result, the channel importance coefficient of each convolution channel is updated based on the obtained first loss parameter, thereby achieving the pruning purpose of the pruning network, compressing the size of the pruning network, and improving the operation speed of the pruning network.

[0079] S209, based on the basic network in the data processing model, perform convolution operations on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple third data features.

[0080] In one implementation, the channel coefficient adjustment device inputs the first multimedia dataset and the second multimedia dataset into the base network of the data processing model. The base network of the data processing model sequentially performs convolution operations on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple third data features. It can be understood that one multimedia data corresponds to one third data feature, and the multimedia data can be any multimedia data in the first multimedia dataset and the second multimedia dataset.

[0081] S210, based on the pruning network in the data processing model, perform convolution operations on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple fourth data features.

[0082] In one implementation, the channel coefficient adjustment device inputs the first multimedia dataset and the second multimedia dataset into the pruning network in the data processing model. The pruning network in the data processing model sequentially performs convolution operations on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple fourth data features. It can be understood that one multimedia data corresponds to one fourth data feature, and the multimedia data can be any multimedia data in the first multimedia dataset and the second multimedia dataset.

[0083] S211, obtain the second loss parameter of the data processing model based on the plurality of third data features and the plurality of fourth data features.

[0084] In one implementation, the channel adjustment device uses the loss function of the data processing model to calculate the second loss parameter based on multiple third data features and multiple fourth data features.

[0085] For example, the loss function can be represented by the following formula:

[0086]

[0087] in, It is any third data feature; It is any fourth data feature; The pseudo-labels are based on the pre-set tasks. Specifically, they are the pseudo-labels corresponding to the multimedia data corresponding to the third data feature and the multimedia data corresponding to the fourth data feature; n is the pre-set edge distance parameter; B is the total number of multimedia data in the first multimedia dataset and the second multimedia dataset; and L is the second loss parameter.

[0088] Assume the first multimedia dataset contains four images A, B, C, and D, and the second multimedia dataset contains four images A', B', C', and D'. Then, through the base network of the data processing model, the third data features corresponding to the aforementioned eight images are sequentially obtained as A1, A'1, B1, B'1, C1, C'1, D1, and D'1. Simultaneously, through the pruning network of the data processing model, the fourth data features corresponding to the aforementioned eight images are sequentially obtained as A2, A'2, B2, B'2, C2, C'2, D2, and D'2. Then, one third data feature A1 is obtained from among the multiple fifth data features, and one sixth data feature A2 is obtained from among the multiple fourth data features. Finally, the loss function is applied... Calculate the loss parameter L1 once, then obtain the new third data feature A'1 and the new fourth data feature A'2, calculate the new loss parameter L2, and repeat the above steps until multiple third data features and multiple fourth data features have been calculated. Then, sum the results based on each sub-loss parameter to obtain the second loss parameter L.

[0089] By performing convolution operations on multiple multimedia data in the first and second multimedia datasets, and then obtaining a second loss parameter based on the convolution operation results, the channel importance coefficients of each convolution channel are updated based on the obtained second loss parameter, thereby achieving the pruning purpose of the pruning network, compressing the size of the pruning network, and improving the operation speed of the pruning network.

[0090] S212, based on the first loss parameter and the second loss parameter, update the channel importance coefficient of the convolution channel in the data processing model.

[0091] In one implementation, the channel coefficient adjustment device backpropagates the first loss parameter and the second loss parameter to the pruning network of the data processing model. Based on the received first loss parameter and second loss parameter, the pruning network updates the channel importance coefficients of multiple convolutional channels in the pruning network.

[0092] S213, Obtain the pruning rate of the pruning network in the data processing model.

[0093] In one implementation, the channel coefficient adjustment device determines the channel importance coefficient of each convolutional channel in the pruning network of the data processing model. Then, among the multiple convolutional channels of the pruning network, it identifies target convolutional channels whose channel importance coefficient is less than or equal to a set coefficient threshold, obtains the number of target convolutional channels, and, based on the number of target convolutional channels and the total number of channels in the multiple convolutional channels of the pruning network, obtains the pruning rate of the pruning network in the data processing model. The obtained pruning rate is then compared with a set pruning rate. Here, the pruning rate indicates the compression status of the pruning network, and the set pruning rate is a pre-set pruning rate threshold, which can be calculated based on the user's requirements for model size. When the pruning rate of the pruning network reaches the set pruning rate, it is determined that the pruning of the network is complete.

[0094] S214, if the pruning rate is greater than or equal to the set pruning rate, then stop training the data processing model and obtain the trained data processing model.

[0095] In one implementation, when the pruning rate is greater than or equal to the set pruning rate, the channel coefficient adjustment device determines that the pruning network has been pruned, and then stops inputting multimedia data into the data processing model to stop training the data processing model and obtain the trained data processing model.

[0096] In one implementation, when the channel coefficient adjustment device acquires the trained data processing model, it can remove the base network from the data processing model, meaning that the data processing model only includes the pruned network obtained through training.

[0097] Specifically, when the channel coefficient adjustment device inputs the multimedia data to be processed into the data processing model to obtain the data processing result for the multimedia data to be processed, the data processing model performs convolution operation on the multimedia data to be processed through a pruned network, obtains the convolution result, and outputs it.

[0098] S215, if the pruning rate is less than the set pruning rate, then a set number of multimedia data is obtained, and the set number of multimedia data is used as the first multimedia dataset. The data expansion processing of the first multimedia dataset using the pre-processing task is then performed to obtain the second multimedia dataset.

[0099] In one implementation, when the pruning rate is less than a set pruning rate, the channel coefficient adjustment device determines that the pruning network has not been pruned. Then, it acquires a set amount of multimedia data and forms a first multimedia dataset based on the set amount of multimedia data. Then, it re-executes the aforementioned pre-processing task to perform data expansion processing on the first multimedia dataset to obtain a second multimedia dataset. In this way, the data processing model is trained based on the first multimedia dataset and the second multimedia dataset.

[0100] By setting a pruning rate, the pruning of the network is stopped when the pruning rate reaches the set pruning rate. This avoids the situation where the accuracy of the convolution results obtained based on the pruning network is too low due to the pruning rate being too high. Thus, while compressing the size of the pruning network, the accuracy of the output data of the data processing model is also guaranteed.

[0101] In this embodiment, the pruning accuracy of the data processing model is improved by increasing the amount of training data. Furthermore, the robustness and generalization of the data processing model are enhanced by adding a second loss parameter acquisition method. This improves the pruning accuracy, robustness, and generalization of the data processing model for multimedia task processing, thereby increasing the accuracy of the output data obtained by the data processing model in different scenarios or tasks. Simultaneously, it reduces the accuracy loss of the pruning network during the pruning process. By setting up a base network and a pruning network, when pruning the convolutional channels in the pruning network, the error between the convolutional results of the pruning network and the base network can be determined based on the convolutional results of the base network. This allows for the determination of the pruning method, i.e., the adjustment method of the channel importance coefficients of each convolutional channel. This ensures the accuracy of the output data of the pruning network while also compressing the pruning network, thus improving its computational speed. In other words, the output results of the pruning network are equally accurate compared to the base network. Furthermore, the pruned network has a smaller network size and faster computation speed. By performing convolution operations on multiple multimedia data points in the first multimedia dataset, and then obtaining a first loss parameter based on the convolution operation results, the channel importance coefficients of each convolution channel are updated based on the obtained first loss parameter, thereby achieving the purpose of pruning the network, compressing its size, and improving its computation speed. By performing convolution operations on multiple multimedia data points in the first and second multimedia datasets, and then obtaining a second loss parameter based on the convolution operation results, the channel importance coefficients of each convolution channel are updated based on the obtained second loss parameter, thereby achieving the purpose of pruning the network, compressing its size, and improving its computation speed. By setting a pruning rate, pruning of the network is stopped when the pruning rate reaches the set pruning rate, avoiding the situation where the accuracy of the convolution results obtained from the pruned network is too low due to an excessively high pruning rate. Thus, while compressing the size of the pruned network, the accuracy of the output data of the data processing model is also guaranteed.

[0102] Please see Figure 3 The diagram below illustrates a method for adjusting channel coefficients according to an embodiment of this application. Figure 3 As shown, the method may include the following steps S301-S314.

[0103] S301, determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task.

[0104] In one implementation, a multimedia task refers to the multimedia task performed by the data processing model, such as image classification, image segmentation, object detection, human pose detection, and speech recognition. It can be understood that image classification, image segmentation, object detection, human pose detection, and speech recognition are different processing types. The data processing model is a neural network model obtained by training an initial neural network model based on the initial training dataset corresponding to the multimedia task. It has multiple convolutional kernels and convolutional channels. Essentially, the data processing model is a pre-trained neural network model that has not undergone pruning; that is, it can generate output data that satisfies the corresponding multimedia task based on the input data, but its data processing time is relatively long.

[0105] The channel coefficient adjustment device first determines the processing type of the multimedia task to be performed by the data processing model, and then generates a pretext task for the data processing model for that processing type.

[0106] S302, the first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset.

[0107] In one implementation, a pretext task is used to generate pseudo-labeled data from a large amount of unlabeled data. The first multimedia dataset includes multiple first multimedia data sets, which can be video data, image data, audio data, etc. Different multimedia task types correspond to different data expansion methods. For example, if the multimedia task type is image data processing, the data expansion method can be image rotation, image cropping, image coloring, image flipping, etc. The data expansion method is used to obtain more multimedia data and form a pair of positive and negative samples with the original multimedia data. The second multimedia dataset includes multiple second multimedia data sets, where each first multimedia data set has a corresponding second multimedia data set. That is, after performing data expansion processing on any first multimedia data set, a second multimedia data set is obtained. The currently selected first multimedia data set and the currently generated second multimedia data set have a corresponding relationship. It can be understood that the corresponding first and second multimedia data sets have the same data source.

[0108] The channel coefficient adjustment device acquires the first multimedia dataset, and then performs data expansion processing on multiple first multimedia data in the first multimedia dataset through the pre-task corresponding to the data processing model to obtain multiple second multimedia data, i.e., the second multimedia dataset.

[0109] For example, suppose the first multimedia dataset includes four images: image A, image B, image C and image D. By performing data expansion processing on the four images in the first multimedia dataset through a pre-task, we obtain image A', image B', image C' and image D', which are four corresponding multiple second multimedia datasets.

[0110] It should be noted that the pseudo-label set for this prerequisite task is... Where p is the multimedia data index of multimedia data 1, and q is the multimedia data index of multimedia data 2. When multimedia data 1 and multimedia data 2 originate from the same multimedia data, When multimedia data 1 and multimedia data 2 originate from different multimedia data sources, Multimedia data 1 and multimedia data 2 can both be any multimedia data from the first multimedia dataset and the second multimedia dataset. For example, pseudo-labels for image A and image A'. =1, pseudo-tags for images A and B =0.

[0111] S303, the first multimedia dataset is input into the data processing model to obtain the first loss parameter of the data processing model.

[0112] In one implementation, the channel coefficient adjustment device inputs a first multimedia dataset into a data processing model, causing the data processing model to acquire multiple first data features based on the first multimedia dataset, and then generate a first loss parameter based on the multiple first data features and the loss function in the data processing model.

[0113] S304, traverse the first multimedia dataset and the second multimedia dataset to obtain target multimedia data in the first multimedia dataset and the second multimedia dataset, wherein the target multimedia data is any multimedia data in the first multimedia dataset and the second multimedia dataset.

[0114] In one implementation, the target multimedia data is either multimedia data from a first multimedia dataset or a second multimedia dataset.

[0115] The channel coefficient adjustment device treats the first multimedia dataset and the second multimedia dataset as a whole, and then traverses the first multimedia dataset and the second multimedia dataset to obtain the target multimedia data in the first multimedia dataset and the second multimedia dataset. Then, the currently obtained target multimedia data is input into the base network and the pruning network in the data processing model respectively.

[0116] It should be noted that the data processing model is a multi-threaded data processing model, which means that the data processing model can simultaneously handle the aforementioned process of obtaining the first loss parameter and the current process of obtaining the second loss parameter.

[0117] S305, based on the basic network in the data processing model, perform convolution operation on the target multimedia data to obtain the fifth data feature.

[0118] In one implementation, the channel coefficient adjustment device performs convolution operations on the target multimedia through the basic network in the data processing model to obtain the fifth data feature.

[0119] S306, Based on the pruning network in the data processing model, perform convolution operation on the target multimedia data to obtain the sixth data feature.

[0120] In one implementation, the channel coefficient adjustment device performs convolution operations on the target multimedia through a pruning network in the data processing model to obtain the sixth data feature.

[0121] It should be noted that when the data processing model performs convolution operations on the target multimedia data, it processes the target multimedia data simultaneously through a pruning network and a basic network, and then obtains the fifth data feature and the sixth data feature respectively.

[0122] S307, Based on the fifth data feature and the sixth data feature, obtain the third loss parameter of the data processing model.

[0123] In one implementation, the general coefficient adjustment device uses the calculation formula for the third loss parameter to calculate the third loss parameter of the data processing model based on the fifth and sixth data features.

[0124] For example, the formula for calculating the third loss parameter can be as follows:

[0125]

[0126] in, It is the fifth data feature; It is the sixth data feature; The pseudo-labels are based on the pre-set tasks. Specifically, they are the pseudo-labels corresponding to the target multimedia data corresponding to the fifth data feature and the target multimedia data corresponding to the sixth data feature; n is the pre-set edge distance parameter; L3 is the third loss parameter.

[0127] S308, if the traversal of the first multimedia dataset and the second multimedia dataset is completed, then the second loss parameter of the data processing model is obtained based on the multiple third loss parameters.

[0128] In one implementation, when the general coefficient adjustment device finishes traversing the first and second multimedia datasets, it sums up the multiple third loss parameters obtained in the aforementioned traversal process to obtain the second loss parameters of the data processing model.

[0129] By acquiring target multimedia data from the first and second multimedia datasets, and then obtaining the fifth and sixth data features through the base network and pruning network respectively, the third loss parameter can be calculated directly based on the fifth and sixth data features after the data features are acquired. This eliminates the need to sequentially acquire the corresponding fifth and sixth data features from the multiple fifth and sixth data features obtained from the first and second multimedia datasets according to the generation order of the multiple fifth data features and the generation data of the multiple sixth data features before calculating the second loss parameter, thus improving the calculation speed of the second loss parameter.

[0130] S309, based on the first loss parameter and the second loss parameter, update the channel importance coefficient of the convolution channel in the data processing model.

[0131] In one implementation, the channel coefficient adjustment device backpropagates the first loss parameter and the second loss parameter to the pruning network of the data processing model. Based on the received first loss parameter and second loss parameter, the pruning network updates the channel importance coefficients of multiple convolutional channels in the pruning network.

[0132] S310, Obtain the network accuracy of the pruning network in the data processing model.

[0133] In one implementation, the network precision of a pruned network refers to the accuracy rate of the pruned network, which represents the accuracy of predictions on positive sample results.

[0134] After updating the channel importance coefficients of each convolutional channel in the pruned network, the channel coefficient adjustment device obtains the network accuracy of the pruned network and then compares the network accuracy with an accuracy threshold. The accuracy threshold refers to the lower limit of the prediction accuracy of the pruned network.

[0135] S311, if the network precision is greater than or equal to the precision threshold, then obtain the pruning rate of the pruning network in the data processing model.

[0136] In one implementation, the channel coefficient adjustment device determines the channel importance coefficient of each convolutional channel in the pruned network of the data processing model when the network accuracy is greater than or equal to an accuracy threshold. Then, among the multiple convolutional channels of the pruned network, it identifies target convolutional channels whose channel importance coefficient is less than or equal to a set coefficient threshold, obtains the number of target convolutional channels, and obtains the pruning rate of the pruned network in the data processing model based on the number of target convolutional channels and the total number of channels in the multiple convolutional channels of the pruned network. The obtained pruning rate is then compared with a set pruning rate. Here, the pruning rate indicates the compression status of the pruned network, and the set pruning rate is a pre-set pruning rate threshold. When the pruning rate of the pruned network reaches the set pruning rate, it is determined that the pruning of the network is complete.

[0137] S312, if the pruning rate is greater than or equal to the set pruning rate, then stop training the data processing model and obtain the trained data processing model.

[0138] In one implementation, when the pruning rate is greater than or equal to the set pruning rate, the channel coefficient adjustment device determines that the pruning network has been pruned, and then stops inputting multimedia data into the data processing model to stop training the data processing model and obtain the trained data processing model.

[0139] In one implementation, when the channel coefficient adjustment device acquires the trained data processing model, it can remove the base network from the data processing model, meaning that the data processing model only includes the pruned network obtained through training.

[0140] Specifically, when the channel coefficient adjustment device inputs the multimedia data to be processed into the data processing model to obtain the data processing result for the multimedia data to be processed, the data processing model performs convolution operation on the multimedia data to be processed through a pruned network, obtains the convolution result, and outputs it.

[0141] S313, if the pruning rate is less than the set pruning rate, then a set number of multimedia data is obtained, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

[0142] In one implementation, when the pruning rate is less than a set pruning rate, the channel coefficient adjustment device determines that the pruning network has not been pruned. Then, it acquires a set amount of multimedia data and forms a first multimedia dataset based on the set amount of multimedia data. Then, it re-executes the aforementioned pre-processing task to perform data expansion processing on the first multimedia dataset to obtain a second multimedia dataset. In this way, the data processing model is trained based on the first multimedia dataset and the second multimedia dataset.

[0143] S314, if the network precision is less than the precision threshold, then a set number of multimedia data is acquired, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

[0144] In one implementation, when the network accuracy is less than the accuracy threshold, the channel coefficient adjustment device determines that the pruning network has not been pruned. Then, it acquires a set number of multimedia data and forms a first multimedia dataset based on the set number of multimedia data. Then, it re-executes the aforementioned pre-processing task to perform data expansion processing on the first multimedia dataset to obtain a second multimedia dataset. The data processing model is then trained based on the first multimedia dataset and the second multimedia dataset.

[0145] It should be noted that as the training of the pruned network progresses, the pruning rate of the pruned network will gradually increase, and more and more convolutional channels will be pruned. The pruning rate of the pruned network will eventually converge to a certain range, while the network accuracy will show a curve that first decreases and then increases.

[0146] By setting a precision threshold, before obtaining the pruning rate of the pruning network, it is determined whether the network precision of the pruning network has reached the precision threshold. Thus, pruning of the pruning network is stopped only when both the network precision and the pruning rate meet the requirements. This avoids the situation where the accuracy of the convolution results obtained based on the pruning network is too low due to the pruning rate being too high. In this way, the size of the pruning network is reduced while ensuring the accuracy of the output data of the data processing model.

[0147] In this embodiment, based on the processing type of the multimedia task, a pre-task for the data processing model is added to expand the data of the first multimedia dataset, thereby obtaining a large amount of second multimedia dataset. Then, the first loss parameter corresponding to the first multimedia dataset is obtained, and the second loss parameter corresponding to the first and second multimedia datasets is also obtained. The channel coefficients of the convolutional channels in the data processing model are updated based on these two loss parameters obtained through synchronous training. This increases the amount of training data for the data processing model, thereby improving the pruning accuracy of the data processing model. By adding a new way to obtain the loss parameter, namely the second loss parameter acquisition method, the robustness and generalization of the data processing model are improved. In turn, by improving the pruning accuracy, robustness, and generalization of the data processing model for the processing type of multimedia tasks, the accuracy of the output data obtained by the data processing model under different scenarios or tasks is improved. At the same time, the accuracy loss of the pruning network of the data processing model during the pruning process is also reduced. The target multimedia data is obtained from the first and second multimedia datasets. Then, the fifth and sixth data features are obtained through the base network and the pruning network, respectively. After obtaining the data features, the third loss parameter can be calculated directly based on the fifth and sixth data features. This eliminates the need to sequentially obtain the corresponding fifth and sixth data features from the multiple fifth and sixth data features obtained from the first and second multimedia datasets, according to the generation order of the fifth and sixth data features, before calculating the second loss parameter, thus improving the calculation speed of the second loss parameter. By setting a precision threshold, the network precision of the pruning network is determined before obtaining the pruning rate. This ensures that the pruning network stops pruning only when both the network precision and pruning rate meet the requirements, thereby avoiding the situation where the accuracy of the convolution results obtained from the pruning network is too low due to an excessively high pruning rate. Thus, while compressing the size of the pruning network, the accuracy of the output data of the data processing model is also guaranteed.

[0148] Please see Figure 4 The diagram below illustrates a method for adjusting channel coefficients according to an embodiment of this application. Figure 4 As shown, the method may include the following steps S401-S412.

[0149] S401, determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task.

[0150] In one implementation, a multimedia task refers to the multimedia task performed by the data processing model, such as image classification, image segmentation, object detection, human pose detection, and speech recognition. It can be understood that image classification, image segmentation, object detection, human pose detection, and speech recognition are different processing types. The data processing model is a neural network model obtained by training an initial neural network model based on the initial training dataset corresponding to the multimedia task. It has multiple convolutional kernels and convolutional channels. Essentially, the data processing model is a pre-trained neural network model that has not undergone pruning; that is, it can generate output data that satisfies the corresponding multimedia task based on the input data, but its data processing time is relatively long.

[0151] The channel coefficient adjustment device first determines the processing type of the multimedia task to be performed by the data processing model, and then generates a pretext task for the data processing model for that processing type.

[0152] S402, the first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset.

[0153] In one implementation, a pretext task is used to generate pseudo-labeled data from a large amount of unlabeled data. The first multimedia dataset includes multiple first multimedia data sets, which can be video data, image data, audio data, etc. Different multimedia task types correspond to different data expansion methods. For example, if the multimedia task type is image data processing, the data expansion method can be image rotation, image cropping, image coloring, image flipping, etc. The data expansion method is used to obtain more multimedia data and form a pair of positive and negative samples with the original multimedia data. The second multimedia dataset includes multiple second multimedia data sets, where each first multimedia data set has a corresponding second multimedia data set. That is, after performing data expansion processing on any first multimedia data set, a second multimedia data set is obtained. The currently selected first multimedia data set and the currently generated second multimedia data set have a corresponding relationship. It can be understood that the corresponding first and second multimedia data sets have the same data source.

[0154] The channel coefficient adjustment device acquires the first multimedia dataset, and then performs data expansion processing on multiple first multimedia data in the first multimedia dataset through the pre-task corresponding to the data processing model to obtain multiple second multimedia data, i.e., the second multimedia dataset.

[0155] For example, suppose the first multimedia dataset includes four images: image A, image B, image C and image D. By performing data expansion processing on the four images in the first multimedia dataset through a pre-task, we obtain image A', image B', image C' and image D', which are four corresponding multiple second multimedia datasets.

[0156] It should be noted that the pseudo-label set for this prerequisite task is... Where p is the multimedia data index of multimedia data 1, and q is the multimedia data index of multimedia data 2. When multimedia data 1 and multimedia data 2 originate from the same multimedia data, When multimedia data 1 and multimedia data 2 originate from different multimedia data sources, Multimedia data 1 and multimedia data 2 can both be any multimedia data from the first multimedia dataset and the second multimedia dataset. For example, pseudo-labels for image A and image A'. =1, pseudo-tags for images A and B =0.

[0157] S403, the first multimedia dataset is input into the data processing model to obtain the first loss parameter of the data processing model.

[0158] In one implementation, the channel coefficient adjustment device inputs a first multimedia dataset into a data processing model, causing the data processing model to acquire multiple first data features based on the first multimedia dataset, and then generate a first loss parameter based on the multiple first data features and the loss function in the data processing model.

[0159] S404, based on the pruning network in the data processing model, perform convolution operations on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple seventh data features.

[0160] In one implementation, the channel coefficient adjustment device inputs the first multimedia dataset and the second multimedia dataset into the pruning network in the data processing model. The pruning network in the data processing model sequentially performs convolution operations on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple seventh data features. It can be understood that one multimedia data corresponds to one seventh data feature, and the multimedia data can be any multimedia data in the first multimedia dataset and the second multimedia dataset.

[0161] S405, obtain the second loss parameter of the data processing model based on the plurality of seventh data features.

[0162] In one implementation, the channel adjustment device uses the loss function of the data processing model to calculate the second loss parameter based on multiple seventh data features.

[0163] The data processing model uses a pruning network to perform convolution operations on multiple multimedia data in the first and second multimedia datasets. Then, based on the convolution operation results, a second loss parameter is obtained. This eliminates the need for the basic network of the data processing model to participate in the calculation of the second loss parameter, thus improving the pruning speed of the data processing model.

[0164] In one implementation, obtaining the second loss parameter of the data processing model based on the plurality of seventh data features includes the following steps:

[0165] Traverse the plurality of seventh data features to obtain any two seventh data features from the plurality of seventh data features;

[0166] The fourth loss parameter of the data processing model is obtained based on any two of the seventh data features;

[0167] If the traversal of the plurality of seventh data features is completed, then the second loss parameter of the data processing model is obtained based on the plurality of fourth loss parameters.

[0168] The channel coefficient adjustment device obtains any two seventh data features from multiple seventh data features. Specifically, this can be achieved by iterating through the seventh data features. For example, if the first multimedia dataset contains four images A, B, C, and D, and the second multimedia dataset contains four images A', B', C', and D', the channel coefficient adjustment device first obtains the data feature corresponding to image A as the first seventh data feature. Then, it sequentially obtains the data features corresponding to images B, C, ..., C', and D' as the second seventh data feature. At this point, the iteration for image A is complete. Then, it obtains the data feature corresponding to image B as the first seventh data feature, and then sequentially obtains the data features corresponding to images C, D, ..., C', and D' as the second seventh data feature, until the iteration of multiple seventh data features is complete. It should be emphasized that the aforementioned iteration method is only an example and does not limit the method of obtaining the seventh data features.

[0169] The channel coefficient adjustment device uses the loss function of the data processing model to calculate the fourth loss parameter based on the two seventh data features obtained above.

[0170] For example, the formula for calculating the fourth loss parameter can be as follows:

[0171]

[0172] in, and It is any two seventh data features; These are pseudo-tags set based on prerequisite tasks; specifically... Corresponding multimedia data and The pseudo-labels corresponding to the first multimedia data; n is a pre-set edge distance parameter; B is the total number of multimedia data in the first and second multimedia datasets; L4 is the fourth loss parameter.

[0173] The channel coefficient adjustment device finally sums up the multiple fourth loss parameters it has acquired, and uses the summation result as the second loss parameter of the data processing model.

[0174] By traversing multiple seventh data features, a fourth loss parameter is calculated for all combinations. Finally, the second loss parameter of the data processing model is obtained based on multiple fourth loss parameters. Without requiring the basic network of the data processing model to participate in the calculation of the second loss parameter, the pruning speed of the data processing model is improved. At the same time, the accuracy of the second loss parameter is improved by increasing the amount of data used to obtain the second loss parameter.

[0175] S406, based on the first loss parameter and the second loss parameter, update the channel importance coefficients of the convolutional channels in the data processing model to obtain the trained data processing model.

[0176] In one implementation, the channel coefficient adjustment device backpropagates the first loss parameter and the second loss parameter to the pruning network of the data processing model. Based on the received first loss parameter and second loss parameter, the pruning network updates the channel importance coefficients of multiple convolutional channels in the pruning network.

[0177] In this embodiment, based on the processing type of the multimedia task, a pre-task for the data processing model is added. This pre-task expands the first multimedia dataset to obtain a large amount of second multimedia dataset. Then, the first loss parameter corresponding to the first multimedia dataset is obtained, along with the second loss parameters corresponding to both the first and second multimedia datasets. The channel coefficients of the convolutional channels in the data processing model are updated based on these two synchronously trained loss parameters. This increases the amount of training data, thereby improving the pruning accuracy of the data processing model. Furthermore, by adding a new path for obtaining the second loss parameter, the robustness and generalization of the data processing model are improved. Ultimately, by enhancing the pruning accuracy, robustness, and generalization of the data processing model for the multimedia task processing type, the model is better suited for different scenarios or scenarios. The accuracy of the output data obtained under the task is improved, and the accuracy loss of the pruning network of the data processing model during the pruning process is also reduced. By performing convolution operations on multiple multimedia data in the first and second multimedia datasets through the pruning network of the data processing model, and then obtaining the second loss parameter based on the convolution operation result, the basic network of the data processing model does not need to participate in the calculation process of the second loss parameter, thus improving the pruning speed of the data processing model. By traversing the multiple seventh data features obtained, a fourth loss parameter is calculated for all combinations. Finally, the second loss parameter of the data processing model is obtained based on multiple fourth loss parameters. Under the premise of improving the pruning speed of the data processing model without the participation of the basic network of the data processing model in the calculation process of the second loss parameter, the accuracy of the second loss parameter is improved by increasing the amount of data used to obtain the second loss parameter.

[0178] The following will be combined with the appendix Figure 5 -Appendix Figure 6 This application provides a detailed description of the channel coefficient adjustment device provided in its embodiments. It should be noted that the appendix... Figure 5 -Appendix Figure 6 Channel coefficient adjustment device, used to perform the present application Figures 1-4 The methods shown in the embodiments are for illustrative purposes only, illustrating the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figures 1-4 The example shown.

[0179] Please see Figure 5 The diagram below illustrates the structure of a channel coefficient adjustment device according to an embodiment of this application. Figure 5 As shown, the channel coefficient adjustment device 2 in this embodiment may include: a task determination module 21, a data processing module 22, a first acquisition module 23, a second acquisition module 24, and a coefficient update module 25.

[0180] Task determination module 21 is used to determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task;

[0181] Data processing module 22 is used to perform data expansion processing on the first multimedia dataset using the aforementioned pre-task to obtain a second multimedia dataset;

[0182] The first acquisition module 23 is used to input the first multimedia dataset into the data processing model to obtain the first loss parameter of the data processing model;

[0183] The second acquisition module 24 is used to input the first multimedia dataset and the second multimedia dataset into the data processing model for training, so as to obtain the second loss parameter of the data processing model;

[0184] The coefficient update module 25 is used to update the channel importance coefficients of the convolutional channels in the data processing model based on the first loss parameter and the second loss parameter, so as to obtain the trained data processing model.

[0185] In this embodiment, based on the processing type of the multimedia task, a pre-task for the data processing model is added to expand the data of the first multimedia dataset, thereby obtaining a large amount of second multimedia dataset. Then, the first loss parameter corresponding to the first multimedia dataset is obtained, and the second loss parameter corresponding to the first and second multimedia datasets is also obtained. The channel coefficients of the convolutional channels in the data processing model are updated based on the loss parameters obtained from these two synchronous trainings. This increases the amount of training data for the data processing model, thereby improving the pruning accuracy of the data processing model. Furthermore, by adding a new path for obtaining the second loss parameter, the robustness and generalization of the data processing model are improved. In turn, by improving the pruning accuracy, robustness, and generalization of the data processing model for the processing type of multimedia tasks, the accuracy of the output data obtained by the data processing model under different scenarios or tasks is improved. At the same time, the accuracy loss of the pruning network of the data processing model during the pruning process is also reduced.

[0186] In one implementation, the first acquisition module 21 is specifically used for:

[0187] Based on the basic network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset to obtain multiple first data features.

[0188] Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset to obtain multiple second data features.

[0189] The first loss parameter of the data processing model is obtained based on the plurality of first data features and the plurality of second data features.

[0190] In one implementation, the second acquisition module 22 is specifically used for:

[0191] Based on the basic network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple third data features;

[0192] Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple fourth data features;

[0193] The second loss parameter of the data processing model is obtained based on the plurality of third data features and the plurality of fourth data features.

[0194] In one implementation, the second acquisition module 22 is specifically used for:

[0195] Traverse the first multimedia dataset and the second multimedia dataset to obtain target multimedia data in the first multimedia dataset and the second multimedia dataset, wherein the target multimedia data is any multimedia data in the first multimedia dataset and the second multimedia dataset;

[0196] Based on the basic network in the data processing model, convolution operation is performed on the target multimedia data to obtain the fifth data feature;

[0197] Based on the pruning network in the data processing model, convolution operation is performed on the target multimedia data to obtain the sixth data feature;

[0198] The third loss parameter of the data processing model is obtained based on the fifth and sixth data features.

[0199] If the first multimedia dataset and the second multimedia dataset have been traversed, then the second loss parameter of the data processing model is obtained based on multiple third loss parameters.

[0200] In one implementation, the second acquisition module 22 is specifically used for:

[0201] Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple seventh data features;

[0202] The second loss parameter of the data processing model is obtained based on the multiple seventh data features.

[0203] In one implementation, the second acquisition module 22 is specifically used for:

[0204] Traverse the plurality of seventh data features to obtain any two seventh data features from the plurality of seventh data features;

[0205] The fourth loss parameter of the data processing model is obtained based on any two of the seventh data features;

[0206] If the traversal of the plurality of seventh data features is completed, then the second loss parameter of the data processing model is obtained based on the plurality of fourth loss parameters.

[0207] In one implementation, please refer to Figure 6 ,like Figure 6 As shown, the channel coefficient adjustment device 2 may further include: a training termination module 26.

[0208] The training termination module 26 is specifically used for:

[0209] Based on the first loss parameter and the second loss parameter, update the channel importance coefficient of the convolutional channels in the data processing model;

[0210] Obtain the pruning rate of the pruning network in the data processing model;

[0211] If the pruning rate is greater than or equal to the set pruning rate, then training the data processing model is stopped, and the trained data processing model is obtained.

[0212] If the pruning rate is less than the set pruning rate, then a set number of multimedia data is acquired, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

[0213] In one implementation, the training termination module 26 is specifically used for:

[0214] Obtain the network accuracy of the pruning network in the data processing model;

[0215] If the network precision is greater than or equal to the precision threshold, obtain the pruning rate of the pruning network in the data processing model;

[0216] If the network precision is less than the precision threshold, a set number of multimedia data is acquired, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

[0217] In one implementation, please refer to Figure 6 ,like Figure 6 As shown, the channel coefficient adjustment device 2 may further include: a model acquisition module 27.

[0218] The model acquisition module 27 is specifically used for:

[0219] Obtain the initial data processing network;

[0220] An initial channel importance coefficient is set in multiple convolutional channels of the initial data processing network to obtain a pruned data processing network;

[0221] The initial data processing network is used as the base network of the data processing model, and the pruned data processing network is used as the pruning network of the data processing model.

[0222] This application embodiment also provides a storage medium that can store multiple program instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1-4 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figures 1-4 The specific details of the illustrated embodiments will not be elaborated here.

[0223] Please see Figure 7 This document provides a schematic diagram of the structure of a computer device according to an embodiment of this application. Figure 7As shown, the computer device 1000 may include: at least one processor 1001, at least one communication bus 1002, at least one input / output interface 1003, at least one network interface 1004, and at least one memory 1005. The processor 1001 may include one or more processing cores. The processor 1001 connects various parts within the computer device 1000 using various interfaces and lines, and performs various functions of the terminal 1000 and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The communication bus 1002 is used to implement communication between these components. Figure 7 As shown, the memory 1005, which serves as a storage medium for a terminal device, may include an operating system, a network communication module, an input / output interface module, and a channel coefficient adjustment program.

[0224] exist Figure 7 In the computer device 1000 shown, the input / output interface 1003 is mainly used to provide an input interface for users and access devices to obtain data input by users and access devices.

[0225] In one embodiment.

[0226] The processor 1001 can be used to call the channel coefficient adjustment program stored in the memory 1005 and specifically perform the following operations:

[0227] The prerequisite tasks for the data processing model are determined based on the processing type of the multimedia task.

[0228] The first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset.

[0229] The first multimedia dataset is input into the data processing model to obtain the first loss parameter of the data processing model;

[0230] The first multimedia dataset and the second multimedia dataset are input into the data processing model for training to obtain the second loss parameter of the data processing model;

[0231] Based on the first loss parameter and the second loss parameter, the channel importance coefficients of the convolutional channels in the data processing model are updated to obtain the trained data processing model.

[0232] Optionally, when the processor 1001 performs the step of inputting the first multimedia dataset into the data processing model to obtain the first loss parameter of the data processing model, it specifically performs the following operations:

[0233] Based on the basic network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset to obtain multiple first data features.

[0234] Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset to obtain multiple second data features.

[0235] The first loss parameter of the data processing model is obtained based on the plurality of first data features and the plurality of second data features.

[0236] Optionally, when the processor 1001 performs the step of inputting the first multimedia dataset and the second multimedia dataset into the data processing model for training to obtain the second loss parameter of the data processing model, it specifically performs the following operations:

[0237] Based on the basic network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple third data features;

[0238] Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple fourth data features;

[0239] The second loss parameter of the data processing model is obtained based on the plurality of third data features and the plurality of fourth data features.

[0240] Optionally, when the processor 1001 performs the step of inputting the first multimedia dataset and the second multimedia dataset into the data processing model for training to obtain the second loss parameter of the data processing model, it specifically performs the following operations:

[0241] Traverse the first multimedia dataset and the second multimedia dataset to obtain target multimedia data in the first multimedia dataset and the second multimedia dataset, wherein the target multimedia data is any multimedia data in the first multimedia dataset and the second multimedia dataset;

[0242] Based on the basic network in the data processing model, convolution operation is performed on the target multimedia data to obtain the fifth data feature;

[0243] Based on the pruning network in the data processing model, convolution operation is performed on the target multimedia data to obtain the sixth data feature;

[0244] The third loss parameter of the data processing model is obtained based on the fifth and sixth data features.

[0245] If the first multimedia dataset and the second multimedia dataset have been traversed, then the second loss parameter of the data processing model is obtained based on multiple third loss parameters.

[0246] Optionally, when the processor 1001 performs the step of inputting the first multimedia dataset and the second multimedia dataset into the data processing model for training to obtain the second loss parameter of the data processing model, it specifically performs the following operations:

[0247] Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple seventh data features;

[0248] The second loss parameter of the data processing model is obtained based on the multiple seventh data features.

[0249] Optionally, when the processor 1001 executes the process of obtaining the second loss parameters of the data processing model based on the plurality of seventh data features, it specifically performs the following operations:

[0250] Traverse the plurality of seventh data features to obtain any two seventh data features from the plurality of seventh data features;

[0251] The fourth loss parameter of the data processing model is obtained based on any two of the seventh data features;

[0252] If the traversal of the plurality of seventh data features is completed, then the second loss parameter of the data processing model is obtained based on the plurality of fourth loss parameters.

[0253] Optionally, when the processor 1001 performs the operation of updating the channel importance coefficients of the convolutional channels in the data processing model based on the first loss parameter and the second loss parameter to obtain the trained data processing model, it specifically performs the following operations:

[0254] Based on the first loss parameter and the second loss parameter, update the channel importance coefficient of the convolutional channels in the data processing model;

[0255] Obtain the pruning rate of the pruning network in the data processing model;

[0256] If the pruning rate is greater than or equal to the set pruning rate, then training the data processing model is stopped, and the trained data processing model is obtained.

[0257] If the pruning rate is less than the set pruning rate, then a set number of multimedia data is acquired, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

[0258] Optionally, before executing the pre-processing task of determining the data processing model based on the multimedia task processing type, the processor 1001 further performs the following operations:

[0259] Obtain the initial data processing network;

[0260] An initial channel importance coefficient is set in multiple convolutional channels of the initial data processing network to obtain a pruned data processing network;

[0261] The initial data processing network is used as the base network of the data processing model, and the pruned data processing network is used as the pruning network of the data processing model.

[0262] Optionally, before executing the pre-processing task of determining the data processing model based on the multimedia task processing type, the processor 1001 further performs the following operations:

[0263] Obtain the initial data processing network;

[0264] An initial channel importance coefficient is set in multiple convolutional channels of the initial data processing network to obtain a pruned data processing network;

[0265] The initial data processing network is used as the base network of the data processing model, and the pruned data processing network is used as the pruning network of the data processing model.

[0266] In this embodiment, based on the processing type of the multimedia task, a pre-task for the data processing model is added to expand the data of the first multimedia dataset, thereby obtaining a large amount of second multimedia dataset. Then, the first loss parameter corresponding to the first multimedia dataset is obtained, and the second loss parameter corresponding to the first and second multimedia datasets is also obtained. The channel coefficients of the convolutional channels in the data processing model are updated based on the loss parameters obtained from these two synchronous trainings. This increases the amount of training data for the data processing model, thereby improving the pruning accuracy of the data processing model. Furthermore, by adding a new path for obtaining the second loss parameter, the robustness and generalization of the data processing model are improved. In turn, by improving the pruning accuracy, robustness, and generalization of the data processing model for the processing type of multimedia tasks, the accuracy of the output data obtained by the data processing model under different scenarios or tasks is improved. At the same time, the accuracy loss of the pruning network of the data processing model during the pruning process is also reduced.

[0267] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0268] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0269] The above is a description of a channel coefficient adjustment method, apparatus, storage medium, and device provided in this application. For those skilled in the art, based on the ideas of the embodiments of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for adjusting channel coefficients, characterized in that, The method includes: The prerequisite tasks for the data processing model are determined based on the processing type of the multimedia task. The first multimedia dataset is expanded using the aforementioned pre-task to obtain the second multimedia dataset. The first multimedia dataset is input into the data processing model to obtain the first loss parameter of the data processing model; the first loss parameter is generated based on multiple first data features and the loss function in the data processing model, wherein the first data features are obtained by the data processing model based on the first multimedia dataset; The first multimedia dataset and the second multimedia dataset are input into the data processing model for training to obtain the second loss parameter of the data processing model; the second loss parameter is generated based on multiple second data features and the loss function in the data processing model, and the second data features are obtained by the data processing model based on the first multimedia dataset and the second multimedia dataset; Based on the first loss parameter and the second loss parameter, the channel importance coefficients of the convolutional channels in the data processing model are updated to obtain the trained data processing model.

2. The method according to claim 1, characterized in that, The step of inputting the first multimedia dataset into the data processing model to obtain the first loss parameter of the data processing model includes: Based on the basic network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset to obtain multiple first data features. Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset to obtain multiple second data features. The first loss parameter of the data processing model is obtained based on the plurality of first data features and the plurality of second data features.

3. The method according to claim 1, characterized in that, The step of inputting the first multimedia dataset and the second multimedia dataset into the data processing model for training to obtain the second loss parameter of the data processing model includes: Based on the basic network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple third data features; Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple fourth data features; The second loss parameter of the data processing model is obtained based on the plurality of third data features and the plurality of fourth data features.

4. The method according to claim 1, characterized in that, The step of inputting the first multimedia dataset and the second multimedia dataset into the data processing model for training to obtain the second loss parameter of the data processing model includes: Traverse the first multimedia dataset and the second multimedia dataset to obtain target multimedia data in the first multimedia dataset and the second multimedia dataset, wherein the target multimedia data is any multimedia data in the first multimedia dataset and the second multimedia dataset; Based on the basic network in the data processing model, convolution operation is performed on the target multimedia data to obtain the fifth data feature; Based on the pruning network in the data processing model, convolution operation is performed on the target multimedia data to obtain the sixth data feature; The third loss parameter of the data processing model is obtained based on the fifth and sixth data features; If the first multimedia dataset and the second multimedia dataset have been traversed, then the second loss parameter of the data processing model is obtained based on multiple third loss parameters.

5. The method according to claim 1, characterized in that, The step of inputting the first multimedia dataset and the second multimedia dataset into the data processing model for training to obtain the second loss parameter of the data processing model includes: Based on the pruning network in the data processing model, convolution operations are performed on multiple first multimedia data in the first multimedia dataset and multiple second multimedia data in the second multimedia dataset to obtain multiple seventh data features; The second loss parameter of the data processing model is obtained based on the multiple seventh data features.

6. The method according to claim 5, characterized in that, The process of obtaining the second loss parameter of the data processing model based on the plurality of seventh data features includes: Traverse the plurality of seventh data features to obtain any two seventh data features from the plurality of seventh data features; The fourth loss parameter of the data processing model is obtained based on any two of the seventh data features; If the traversal of the plurality of seventh data features is completed, then the second loss parameter of the data processing model is obtained based on the plurality of fourth loss parameters.

7. The method according to any one of claims 1-6, characterized in that, The step of updating the channel importance coefficients of the convolutional channels in the data processing model based on the first loss parameter and the second loss parameter to obtain the trained data processing model includes: Based on the first loss parameter and the second loss parameter, update the channel importance coefficient of the convolutional channels in the data processing model; Obtain the pruning rate of the pruning network in the data processing model; If the pruning rate is greater than or equal to the set pruning rate, then training the data processing model is stopped, and the trained data processing model is obtained. If the pruning rate is less than the set pruning rate, then a set number of multimedia data is acquired, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

8. The method according to claim 7, characterized in that, Before obtaining the pruning rate of the pruning network in the data processing model, the method further includes: Obtain the network accuracy of the pruning network in the data processing model; If the network accuracy is greater than or equal to the accuracy threshold, obtain the pruning rate of the pruning network in the data processing model; If the network precision is less than the precision threshold, a set number of multimedia data is acquired, and the set number of multimedia data is used as the first multimedia dataset. The first multimedia dataset is then expanded using the pre-processing task to obtain the second multimedia dataset.

9. The method according to claim 1, characterized in that, Before determining the prerequisite tasks for the data processing model based on the processing type of the multimedia task, the following steps are also included: Obtain the initial data processing network; An initial channel importance coefficient is set in multiple convolutional channels of the initial data processing network to obtain a pruned data processing network; The initial data processing network is used as the base network of the data processing model, and the pruned data processing network is used as the pruning network of the data processing model.

10. A channel coefficient adjustment device, characterized in that, include: The task determination module is used to determine the prerequisite tasks for the data processing model based on the processing type of the multimedia task. The data processing module is used to perform data expansion processing on the first multimedia dataset using the aforementioned pre-task to obtain the second multimedia dataset. The first acquisition module is used to input the first multimedia dataset into the data processing model to obtain the first loss parameter of the data processing model; the first loss parameter is generated based on multiple first data features and the loss function in the data processing model, wherein the first data features are acquired by the data processing model based on the first multimedia dataset; The second acquisition module is used to input the first multimedia dataset and the second multimedia dataset into the data processing model for training, so as to obtain the second loss parameter of the data processing model; the second loss parameter is generated based on multiple second data features and the loss function in the data processing model, wherein the second data features are obtained by the data processing model based on the first multimedia dataset and the second multimedia dataset; The coefficient update module is used to update the channel importance coefficients of the convolutional channels in the data processing model based on the first loss parameter and the second loss parameter, so as to obtain the trained data processing model.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the channel coefficient adjustment method according to any one of claims 1-9.

12. A computer device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the channel coefficient adjustment method as described in any one of claims 1-9.

Images