Task processing model generation methods, devices, electronic devices, and storage media

Abstract

This disclosure relates to a method, apparatus, electronic device, and storage medium for generating a task processing model. The method includes: acquiring an initial channel feature set output by a task model to be processed; using the initial channel feature set as the current processing feature set; determining a first channel feature whose feature length information satisfies a preset feature length condition and the current channel feature set from the current processing feature set; determining whether there exists a second channel feature in the current channel feature set whose feature similarity to the first channel feature satisfies a preset similarity condition; if so, updating the current channel feature set and using the updated current channel feature set as the current processing feature set. After repeating the above steps, a target channel feature set including the first channel feature is output. This method can save channels with high channel response intensity in the task model to be processed and remove channels that are redundant with the channel, thereby improving the efficiency of multimedia resource processing and balancing task processing accuracy and system resource consumption.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to methods, apparatus, electronic devices, and storage media for generating task processing models. Background Technology

[0002] In recent years, deep neural networks have been widely used in various computer vision tasks, including image classification, object detection, and image segmentation. While increasing network depth improves recognition performance, it also requires enormous computational resources. One approach is to prune neural networks based on prior human knowledge, removing redundant structures. However, this method relies on prior knowledge and fails to consider the network structure, leading to low efficiency in multimedia resource processing. Summary of the Invention

[0003] This disclosure provides a method, apparatus, electronic device, and storage medium for generating task processing models, to at least address the problem of low efficiency in multimedia resource processing in related technologies. The technical solution of this disclosure is as follows:

[0004] According to a first aspect of the present disclosure, a method for generating a task processing model is provided, the method comprising:

[0005] The initial channel feature set output by the task model to be processed is used as the current processing feature set; the initial channel feature set includes the resource features of the preset multimedia resources in multiple channels of the task model to be processed, which are output by the task model to be processed after performing task processing on the preset multimedia resources.

[0006] The current processing feature set is divided into a first channel feature set and a current channel feature set; the first channel feature set consists of resource features in the channel whose feature modulus information satisfies a preset modulus condition.

[0007] The feature comparison is performed on each resource feature in the first channel feature set and the current channel feature set to obtain the feature comparison result;

[0008] If the feature comparison result indicates that a second channel feature exists in the current channel feature set, the second channel feature is deleted from the current channel feature set;

[0009] The current channel feature set after deleting the second channel feature is taken as the current processing feature set;

[0010] Repeat the steps from dividing the current processing feature set into the first channel feature set and the current channel feature set to using the current channel feature set after deleting the second channel feature set as the current processing feature set, until the current processing feature set is an empty set, and combine the first channel features determined each time to form the target channel feature set;

[0011] Based on the target channel feature set, the task model to be processed is pruned to obtain the target task processing model.

[0012] As an optional embodiment, dividing the current processing feature set into a first channel feature set and a current channel feature set includes:

[0013] Determine the feature modulus information of each resource feature in the current processing feature set;

[0014] The maximum value in the feature modulus information is determined as the target modulus information;

[0015] The resource features corresponding to the target module length information are used as the first channel features;

[0016] The current channel feature set is composed of the resource features other than the first channel feature in the current processing feature set.

[0017] As an optional embodiment, determining the feature modulus information of each resource feature in the current processing feature set includes:

[0018] For each resource feature in the current processing feature set, feature normalization processing is performed to obtain the feature modulus information.

[0019] As an optional embodiment, the preset similarity condition is that the feature similarity is greater than or equal to a preset similarity threshold, and the feature comparison between the first channel feature and the current channel feature set to obtain the feature comparison result includes:

[0020] Determine the feature similarity between the first channel feature and each resource feature in the current channel feature set;

[0021] The feature similarity is compared with a similarity threshold to obtain the feature comparison result.

[0022] As an optional embodiment, before using the initial channel feature set output by the task model as the current processing feature set, the method further includes:

[0023] Obtain the initial resource features corresponding to each channel in the network layer of the task model to be processed;

[0024] The initial resource features are subjected to feature pooling to obtain the resource pooling features corresponding to each channel;

[0025] The resource pooling features corresponding to each channel are combined to form the initial channel feature set.

[0026] As an optional embodiment, the step of pruning the task model to be processed based on the target channel feature set to obtain the target task processing model includes:

[0027] In the task-to-be-processed model, determine the target channel corresponding to each resource feature in the target channel feature set;

[0028] Based on the target channel, the channels in the task-to-be-processed model are pruned to obtain the target task processing model.

[0029] According to a second aspect of the present disclosure, a task processing method is provided, the method comprising:

[0030] Acquire multimedia resources to be processed;

[0031] The multimedia resources to be processed are input into the target task processing model generated according to the task processing model generation method described above, and the task processing result is obtained.

[0032] According to a third aspect of the present disclosure, a task processing model generation apparatus is provided, the apparatus comprising:

[0033] The current processing feature set determination module is configured to use the initial channel feature set output by the task model to be processed as the current processing feature set; the initial channel feature set includes the resource features of the preset multimedia resources in multiple channels of the task model to be processed, which are output by the task model to be processed after performing task processing on the preset multimedia resources.

[0034] The feature set partitioning module is configured to partition the current processed feature set into a first channel feature and a current channel feature set; the first channel feature is a resource feature in a channel whose feature modulus information satisfies a preset modulus condition.

[0035] The feature comparison module is configured to perform feature comparison on each resource feature in the first channel feature set and the current channel feature set to obtain feature comparison results;

[0036] The feature deletion module is configured to delete the second channel feature from the current channel feature set when the feature comparison result indicates that the current channel feature set contains a second channel feature.

[0037] The current processing feature set update module is configured to use the current channel feature set after deleting the second channel feature as the current processing feature set;

[0038] The loop processing module is configured to repeatedly perform the steps of dividing the current processing feature set into a first channel feature and a current channel feature set to using the current channel feature set after deleting the second channel feature as the current processing feature set, until the current processing feature set is an empty set, and to combine the first channel features determined each time into a target channel feature set;

[0039] The pruning module is configured to perform pruning on the task model to be processed based on the target channel feature set, so as to obtain the target task processing model.

[0040] As an optional embodiment, the feature set partitioning module includes:

[0041] The feature modulus determination unit is configured to determine the feature modulus information of each resource feature in the current processing feature set.

[0042] The target modulus determination unit is configured to determine the maximum value in the feature modulus information as the target modulus information;

[0043] The first channel feature determination unit is configured to use the resource feature corresponding to the target modulus information as the first channel feature.

[0044] The current channel feature set determination unit is configured to perform the operation of forming the current channel feature set by combining the other channel features in the current processing feature set, excluding the first channel feature.

[0045] As an optional embodiment, the feature modulus determination unit includes:

[0046] The normalization processing unit is configured to perform feature normalization processing on each resource feature in the current processing feature set to obtain the feature modulus information.

[0047] As an optional embodiment, the preset similarity condition is that the feature similarity is greater than or equal to a preset similarity threshold, and the feature comparison module includes:

[0048] The feature similarity determination unit is configured to determine the feature similarity between the first channel feature and each resource feature in the current channel feature set;

[0049] The similarity comparison unit is configured to compare the feature similarity with the similarity threshold to obtain the feature comparison result.

[0050] As an optional embodiment, the apparatus further includes:

[0051] The initial resource feature acquisition module is configured to acquire the initial resource features corresponding to each channel in the network layer of the task model to be processed.

[0052] The feature pooling module is configured to perform feature pooling processing on the initial resource features to obtain resource pooled features corresponding to each channel; and to form the initial channel feature set by combining the resource pooled features corresponding to each channel.

[0053] As an optional embodiment, the pruning module includes:

[0054] The target channel determination unit is configured to determine the target channel corresponding to each resource feature in the target channel feature set in the task to be processed model.

[0055] The target task processing model determination unit is configured to perform pruning processing on the channels in the task model to be processed based on the target channel to obtain the target task processing model.

[0056] According to a fourth aspect of the present disclosure, a task processing apparatus is provided, the apparatus comprising:

[0057] The pending resource acquisition module is configured to acquire pending multimedia resources;

[0058] The task processing module is configured to input the multimedia resources to be processed into the target task processing model generated according to the task processing model generation method described above, and to obtain the task processing result.

[0059] According to a fifth aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0060] processor;

[0061] Memory used to store the processor's executable instructions;

[0062] The processor is configured to execute the instructions to implement the task processing model generation method or the task processing method described above.

[0063] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the task processing model generation method or the task processing method described above.

[0064] According to a seventh aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, implement the task processing model generation method or task processing method described above.

[0065] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0066] The method obtains the initial channel feature set output by the model of the task to be processed, and uses this initial channel feature set as the current processing feature set. From the current processing feature set, it determines the first channel feature whose feature length information meets a preset condition, and the current channel feature set. It also determines whether there exists a second channel feature in the current channel feature set whose feature similarity to the first channel feature meets a preset similarity condition. If so, it updates the current channel feature set and uses the updated current channel feature set as the current processing feature set. After repeating the above steps, the output is a target channel feature set including the first channel feature. This method can save channels with high channel response intensity in the model of the task to be processed and remove redundant channels, improving the accuracy and effectiveness of pruning, thereby improving the efficiency of multimedia resource processing. It also balances task processing accuracy and system resource consumption during data processing, reducing system resource consumption during data processing while ensuring the task processing accuracy of the pruned model.

[0067] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0068] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0069] Figure 1 This is a schematic diagram illustrating an application scenario of a task processing model generation method according to an exemplary embodiment.

[0070] Figure 2 This is a flowchart illustrating a task processing model generation method according to an exemplary embodiment.

[0071] Figure 3 This is a flowchart illustrating a task processing model generation method for obtaining an initial channel feature set according to an exemplary embodiment.

[0072] Figure 4 This is a flowchart illustrating the division of the current processing feature set in a task processing model generation method according to an exemplary embodiment.

[0073] Figure 5This is a flowchart illustrating feature comparison in a task processing model generation method according to an exemplary embodiment.

[0074] Figure 6 This is a flowchart illustrating pruning in a task processing model generation method according to an exemplary embodiment.

[0075] Figure 7 This is a flowchart illustrating a task processing method according to an exemplary embodiment.

[0076] Figure 8 This is a block diagram illustrating a task processing model generation apparatus according to an exemplary embodiment.

[0077] Figure 9 This is a block diagram illustrating a task processing apparatus according to an exemplary embodiment.

[0078] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment (general structure of a server). Detailed Implementation

[0079] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0080] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0081] Figure 1This is a schematic diagram illustrating an application scenario of a task processing model generation method according to an exemplary embodiment. The application scenario includes a model generation server 110, a task processing server 120, and a client 130. The client 130 sends preset multimedia resources to the model generation server 110 and obtains an initial channel feature set based on the task model to be processed. The model generation server 110 uses the initial channel feature set as the current processing feature set and determines a first channel feature and the current channel feature set from it. The model generation server 110 compares the resource features in the first channel feature set and the current channel feature set to obtain a feature comparison result. If the feature comparison result indicates the existence of a second channel feature in the current channel feature set, the model generation server 110 updates the current channel feature set and updates the current processing feature set based on the updated current channel feature set. The model generation server 110 repeats the above execution steps and outputs a target channel feature set composed of the first channel features. The model generation server 110 performs model pruning on the task model to be processed based on the target channel feature set to obtain the target task processing model.

[0082] The model generation server 110 sends the target task processing model to the task processing server 120. The client 130 sends the multimedia resources to be processed to the task processing server 120. The task processing server 120 processes the multimedia resources to be processed based on the target task processing model and obtains the task processing result.

[0083] In this embodiment, the client 130 includes physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, and smart wearable devices, and may also include software running on the physical device, such as applications. The operating system running on the physical device in this embodiment may include, but is not limited to, Android, iOS, Linux, Unix, and Windows.

[0084] In this embodiment of the disclosure, the model generation server 110 or the task processing server 120 may include a standalone server, a distributed server, or a server cluster consisting of multiple servers. The model generation server 110 or the task processing server 120 may include a network communication unit, a processor, and a memory, etc.

[0085] Figure 2 This is a flowchart illustrating a task processing model generation method according to an exemplary embodiment, such as... Figure 2 As shown, this method is used in a server and includes the following steps.

[0086] S210. Use the initial channel feature set output by the task model to be processed as the current processing feature set; the initial channel feature set includes the output of the task model to be processed after performing task processing on the preset multimedia resources, and the preset multimedia resources in the task model to be processed.

[0087] Resource characteristics in multiple channels of the task management model;

[0088] As an optional embodiment, the preset multimedia resources may include text resources, image resources, video resources, or audio resources, and the task model to be processed may be a model that performs any type of task such as recognition, synthesis, or classification.

[0089] The task model to be processed is a trained model. Preset multimedia resources are input into the task model for task processing, and the resource characteristics of the preset multimedia resources in multiple channels of the task model are obtained during the task processing.

[0090] The initial channel feature set is obtained by constructing the feature set.

[0091] As an optional embodiment, please refer to Figure 3 The initial channel feature set output by the model of the task to be processed is used as...

[0092] Before processing the current feature set, the method also includes:

[0093] S310. Obtain the initial resource features corresponding to each channel in the network layer of the model of the task to be processed;

[0094] 5S320. Perform feature pooling on the initial resource features to obtain the resource pooling features corresponding to each channel;

[0095] S330. Combine the resource pooling features corresponding to each channel into an initial channel feature set.

[0096] As an optional embodiment, the task model to be processed may include multiple network layers. The output of the network layer is obtained by fusing the features of the channel outputs of multiple network layers. The network layer may be a convolutional network, and the number of channels in the convolutional network is the number of convolutional kernels. The initial resource features corresponding to each channel in the network layer of the task model to be processed are obtained.

[0097] 0. By performing feature pooling on the initial resource features, we can obtain the resource pooled features corresponding to each channel. Feature Pool

[0098] The optimization process can be average pooling, which calculates the average value of the values ​​within the feature convolution window corresponding to the initial resource features to obtain the resource pooling feature for each channel. For example, when the feature convolution window is 2x2, the average value of the four values ​​included in the 2x2 window is obtained as the resource pooling feature.

[0099] When the multimedia resource is an image resource or a video resource, the initial resource feature is a feature map. Therefore, the value in feature convolution window 5 is a pixel. This allows us to calculate the average value of each pixel in the feature convolution window corresponding to the initial resource feature, thus obtaining the value for each...

[0100] Resource pooling features corresponding to each channel. The resource pooling features corresponding to each channel can be used as the initial channel feature set.

[0101] Feature pooling can also be performed using max pooling. Max pooling calculates the maximum value within the feature convolution window corresponding to each channel's vinegar feature, obtaining the resource-pooled feature for each channel. For example, in the case of vinegar...

[0102] When the convolution window is a 2x2 window, the maximum value among the four values ​​included in the 2x2 window is taken as the resource pooling feature. By using mean pooling, the overall resource features corresponding to multimedia resources can be preserved. This simplifies the feature similarity comparison process while ensuring the integrity of information in the resource features, thus improving the efficiency of feature similarity comparison while maintaining its accuracy.

[0103] S220. Divide the current processing feature set into a first channel feature set and a current channel feature set; the first channel feature set consists of resource features in the channel whose feature modulus information satisfies the preset modulus condition;

[0104] As an optional embodiment, the feature modulus information can represent the intensity of the channel response. A greater channel response intensity indicates a higher importance of the channel to the task processing result, while a smaller channel response intensity indicates a lower importance of the channel to the task processing result. Resource features from channels whose feature modulus information satisfies the preset modulus condition are selected as the first channel features, while resource features from channels that do not satisfy the preset modulus condition are selected as the current channel feature set.

[0105] As an optional embodiment, please refer to Figure 4 The current feature set is divided into the first channel feature set and the current channel feature set, including:

[0106] S410. Determine the feature modulus information of each resource feature in the current processing feature set;

[0107] S420. Determine the maximum value in the feature modulus information as the target modulus information;

[0108] S430. Use the resource features corresponding to the target module length information as the first channel features;

[0109] S440. Take the resource features other than the first channel features in the current processing feature set as the current channel feature set.

[0110] As an optional embodiment, after determining the feature modulus information of each resource feature in the current processing feature set, the maximum value in the feature modulus information is obtained, and this maximum value is determined as the target modulus information. The resource feature in the channel corresponding to the target modulus information is then used as the first channel feature. During the model generation process, the same operation is used to determine the target modulus information for each updated current processing feature set, that is, the resource feature corresponding to the maximum value of the feature modulus information in each current processing feature set is obtained as the first channel feature.

[0111] The feature modulus information of each resource feature in the initial channel feature set can be determined. This feature modulus information is used as the initial feature modulus set. During model generation, the maximum value among the currently retained feature modulus information in the initial feature modulus set is obtained and used as the target modulus information. Based on this target modulus information, the first channel feature and the current channel feature set are determined in the current processed feature set. The target modulus information is then deleted from the current channel feature set. When the current channel feature set is updated, the initial feature modulus set after deleting the target modulus information is updated in the same way; that is, the feature modulus information corresponding to the second channel feature is deleted from the initial feature modulus set after deleting the target modulus information.

[0112] After determining the feature modulus information of each resource feature in the current processing feature set, the minimum value of the feature modulus information is obtained. This minimum value is then determined as the target modulus information, and the resource feature in the channel corresponding to the target modulus information is used as the first channel feature. During model generation, the same operation is used to determine the target modulus information for each updated current processing feature set, that is, the resource feature corresponding to the minimum value of the feature modulus information in each current processing feature set is obtained as the first channel feature.

[0113] Using the maximum or minimum value in the feature modulus information as the target modulus information allows for the differentiation between channels with high and low channel response strengths. This ensures that the channel with the highest response strength is retained in subsequent pruning operations, preventing the accidental removal of channels that would significantly impact the task processing results and thus guaranteeing the model's task processing accuracy after pruning.

[0114] As an optional embodiment, determining the feature modulus information of each resource feature in the current processing feature set includes:

[0115] For each resource feature in the current feature set, feature normalization is performed to obtain feature modulus information.

[0116] As an optional embodiment, feature normalization is performed on each resource feature in the current processing feature set, so that each resource feature in the current processing feature set is within the same scale interval, thereby obtaining feature magnitude information. Feature normalization can be performed on each resource feature in the current processing feature set based on a preset feature norm to obtain feature magnitude information. For example, the L2 feature norm or other feature norms that can be normalized can be used. Based on the L2 feature norm, the ratio between each resource feature in the current processing feature set and the square root of the sum of squares of the resource features in the current processing feature set can be calculated, thereby normalizing each resource feature in the current processing feature set to the [0, 1] interval and obtaining feature magnitude information.

[0117] Normalizing resource features allows for the measurement of resource features in channels on the same scale. This ensures that when determining the resource feature with the strongest response, the modulus of each resource feature adopts a uniform standard, which can improve the reliability of screening strong response channels and thus improve the accuracy of pruning.

[0118] S230. Perform feature comparison on each resource feature in the first channel feature set and the current channel feature set to obtain the feature comparison results;

[0119] As an optional embodiment, by comparing the first channel feature with each resource feature in the current channel feature set, it can be determined whether there is a redundant second channel feature in the current channel feature set. The feature modulus information of the second channel feature is smaller than the target modulus information corresponding to the first channel feature, but the feature similarity between the second channel feature and the first channel feature meets the preset similarity condition, that is, the channel response intensity of the second channel feature is small and it is redundant with the first channel feature.

[0120] S240. If the feature comparison result indicates that a second channel feature exists in the current channel feature set, delete the second channel feature from the current channel feature set;

[0121] S250. Use the current channel feature set after deleting the second channel feature set as the current processing feature set;

[0122] As an optional embodiment, if a second channel feature exists, the second channel feature can be deleted from the current channel feature set, and the current channel feature set after deleting the second channel feature can be used as the current processing feature set. If no second channel feature exists, the current channel feature set can be used directly as the current processing feature set without updating it.

[0123] As an optional embodiment, please refer to Figure 5The preset similarity condition is that the feature similarity is greater than or equal to a preset similarity threshold. Feature comparison is performed between the first channel feature set and the current channel feature set, and the feature comparison results include:

[0124] S510. Determine the feature similarity between each resource feature in the first channel feature set and the current channel feature set;

[0125] S520. Compare the feature similarity with the similarity threshold to obtain the feature comparison result.

[0126] As an optional embodiment, feature similarity can be the distance between the first channel feature and each resource feature in the current channel feature set. The closer the distance, the greater the feature similarity; the farther the distance, the smaller the feature similarity. This feature similarity can be cosine distance, Euclidean distance, etc. When the feature similarity is cosine distance, the cosine value between the vector of the first channel feature and the vector of each resource feature in the current channel feature set can be calculated, and the degree of similarity between the vectors can be measured based on this cosine value. The range of cosine distance is [-1, 1], where -1 indicates that the vector of the first channel feature is negatively correlated with a certain resource feature in the current channel feature set, 0 indicates that the vector of the first channel feature is independent of a certain resource feature in the current channel feature set, and 1 indicates that the vector of the first channel feature is positively correlated with a certain resource feature in the current channel feature set.

[0127] By comparing feature similarity with a similarity threshold, the feature comparison result can be obtained. If the feature similarity is greater than or equal to the preset similarity threshold, the feature similarity satisfies the preset similarity condition, and the resource feature corresponding to this feature similarity is the second channel feature. The second channel feature is any resource feature in the current channel feature set whose feature similarity to a first channel feature satisfies the preset similarity condition. The current channel feature set may include one or more second channel features. If the feature similarity is less than the preset similarity threshold, the feature similarity does not satisfy the preset similarity condition, and the resource feature corresponding to this feature similarity cannot be used as a second channel feature.

[0128] By setting a feature similarity threshold and comparing the feature similarity with the threshold, the second channel features with a feature similarity greater than the threshold are filtered out and deleted from the current channel feature set. This can remove redundant resource features in the pruning process, thereby improving the accuracy of pruning.

[0129] S260. Repeat the steps of dividing the current processing feature set into the first channel feature and the current channel feature set until the current channel feature set after deleting the second channel feature is used as the current processing feature set, until the current processing feature set is an empty set, and combine the first channel features determined each time to form the target channel feature set;

[0130] As an optional embodiment, by repeatedly executing the steps of dividing the current processing feature set into the first channel feature set and the current channel feature set, and then using the current channel feature set after deleting the second channel feature set as the current processing feature set, resource features with high channel response intensity can be screened from the current processing feature set, and redundant resource features can be removed, thereby obtaining the target channel feature set.

[0131] S270. Based on the target channel feature set, prune the model of the task to be processed to obtain the target task processing model.

[0132] As an optional embodiment, based on the target channel feature set, channels in the task model to be processed are filtered, and the task model to be processed is pruned to obtain the target task processing model.

[0133] As an optional embodiment, please refer to Figure 6 Based on the target channel feature set, the model for the task to be processed is pruned to obtain the target task processing model, which includes:

[0134] S610. In the task model to be processed, determine the target channel corresponding to each resource feature in the target channel feature set;

[0135] S620. Based on the target channel, prune the channels in the task model to be processed to obtain the target task processing model.

[0136] As an optional embodiment, when pruning the model of the task to be processed, a target channel is determined from the model of the task to be processed through the target channel feature set. That is, the channel in the model of the task to be processed that matches the resource features of the target channel feature set is obtained, and the matching channel is taken as the target channel. When the target channel includes the channel with the larger channel response among two similar channels, other channels other than the target channel are deleted from the model of the task to be processed, while the target channel is retained. The network structure of the model of the task to be processed is updated to obtain the target task processing model.

[0137] When the target channel includes two similar channels with smaller channel responses, the target channel is deleted from the task model to be processed, while other channels are retained. The network structure of the task model to be processed is then updated to obtain the target task processing model.

[0138] The target task processing model has a sparser network structure than the task-to-be-processed model, while maintaining the same model accuracy. The target task processing model retains the channels with high response strength from the task-to-be-processed model, and these channels do not contain any redundant channels.

[0139] Based on the target channel corresponding to each resource feature in the target channel feature set, channel filtering can be performed on the model to be processed, thereby retaining channels with high response intensity and ensuring that there are no redundant channels among the retained channels. This balances the task processing accuracy and the system resource consumption during data processing, reducing the system resource consumption during data processing while ensuring the task processing accuracy of the pruned model.

[0140] Figure 7 This is a flowchart illustrating a task processing method according to an exemplary embodiment, such as... Figure 7 As shown, this method is used in a server and includes the following steps:

[0141] S710. Obtain multimedia resources to be processed;

[0142] S720. Input the multimedia resources to be processed into the target task processing model generated according to the task processing model generation method described above, and process the task to obtain the task processing result.

[0143] As an optional embodiment, a multimedia resource to be processed is acquired. This multimedia resource can be text, image, video, or audio resources, etc. The multimedia resource to be processed is then input into the target task processing model obtained after model pruning for task processing, thereby obtaining the task processing result. In the target task processing model, feature fusion is performed on the initial resource features corresponding to the target channel in the network layer to obtain the multimedia resource features output by the network layer. Task processing is then performed based on the multimedia resource features output by the network layer to obtain the task processing result. For example, object recognition, object classification, and image segmentation tasks can be performed based on the multimedia resource features output by the network layer, respectively, to obtain object recognition results, object classification results, and image segmentation results.

[0144] The target task processing model obtained after pruning is used to perform task processing on the multimedia resources to be processed. Thus, the target task processing model can improve the efficiency of task processing while maintaining the accuracy of task processing.

[0145] As an optional embodiment, the above task processing model generation method can be applied to model pruning of any convolutional neural network model. Preset multimedia resources are obtained and used as the dataset for model input. These resources are then input into the task model to be processed, and the initial resource features corresponding to multiple channels in the network layers of the task model are obtained during the task processing. If the network model of the task model has L layers {l1, l2, ..., l...} L}, Preset multimedia resources include When extracting features from each layer of the network, the feature output by the a-th layer is F. a ∈R n×c×h×w Where n represents the number of samples of the preset multimedia resources input, c represents the number of channels in the network layer, and h and w represent the feature parameters, such as height and width when the feature is an image feature. Through mean pooling, the pooling feature M of the a-th network layer can be obtained. a ∈R n×c The specific formula is as follows:

[0146] M a =Pool(F a ) Formula 1

[0147] Here, Pool() represents the pooling operation. In the case of the a-th network layer with c channels, the resource pooling feature corresponding to each channel is M. a,j ∈R n×1 ,j∈{1,2,...,c}. The resource pooling features corresponding to multiple channels in each network layer are used as the initial channel feature set.

[0148] When generating the task processing model, the initial channel feature set includes M. a ={M a,1 M a,2 ,...,M a,c The initial channel feature set is used as the current processing feature set. Normalization is performed on each resource feature in the initial channel feature set to obtain the feature magnitude information of each resource feature in the initial channel feature set. This feature magnitude information is then used as the initial feature magnitude set, which includes E = {e1, e2, ..., e...}. c Initialize the target channel feature set D = {}.

[0149] Using the initial channel feature set as the current processing feature set, and if the current processing feature set is not empty, the following steps are executed in a loop:

[0150] The maximum or minimum value is obtained from the initial feature set as the target modulus information. The first channel feature corresponding to the target modulus information is obtained from the current processing feature set. The first channel feature is removed from the current processing feature set to obtain the current channel feature set. The first channel feature is then placed into the target channel feature set.

[0151] The feature similarity between the first channel feature and each resource feature in the current channel feature set is determined. This feature similarity can be obtained by calculating the cosine distance. The feature similarity is then compared with a preset similarity threshold to obtain the feature comparison result. If the feature comparison result indicates that the feature similarity is greater than the similarity threshold, the resource feature in the current channel feature set corresponding to the feature comparison result indicating that the feature similarity is greater than the similarity threshold is used as the second channel feature.

[0152] Delete the second channel feature from the current channel feature set, and delete the feature modulus information corresponding to the second channel feature from the initial feature modulus set after deleting the target modulus information.

[0153] The current channel feature set after deleting the second channel feature set is used as the current processing feature set.

[0154] If the feature comparison result indicates that the feature similarity is less than the similarity threshold, the current channel feature set can be directly used as the current processing feature set to execute the next loop.

[0155] After the loop ends, the target channel feature set is output, which includes all the first channel features obtained by filtering.

[0156] Based on the target channel feature set, model pruning is performed on the task model to be processed. If the maximum value is obtained from the initial feature magnitude set as the target magnitude information, the target channel corresponding to each resource feature in the target channel feature set is retained in the task model to be processed, while other channels are deleted, resulting in the target task processing model. If the minimum value is obtained from the initial feature magnitude set as the target magnitude information, the target channel corresponding to each resource feature in the target channel feature set is deleted from the task model to be processed, while other channels are retained, resulting in the target task processing model.

[0157] This disclosure proposes a task processing model generation method. The method includes: acquiring an initial channel feature set output by the task model to be processed; using the initial channel feature set as the current processing feature set; determining a first channel feature whose feature length information satisfies a preset feature length condition and the current channel feature set from the current processing feature set; determining whether a second channel feature exists in the current channel feature set whose feature similarity to the first channel feature satisfies a preset similarity condition; if so, updating the current channel feature set and using the updated current channel feature set as the current processing feature set. After repeating the above steps, a target channel feature set including the first channel feature is output. This method can save channels with high channel response intensity in the task model to be processed and remove redundant channels, improving the accuracy and effectiveness of pruning, thereby improving the efficiency of multimedia resource processing and balancing task processing accuracy and system resource consumption during data processing. It reduces system resource consumption during data processing while ensuring the task processing accuracy of the pruned model.

[0158] Figure 8 This is a block diagram illustrating a task processing model generation apparatus according to an exemplary embodiment. (Refer to...) Figure 8 The device includes:

[0159] The current processing feature set determination module 810 is configured to use the initial channel feature set output by the task model to be processed as the current processing feature set; the initial channel feature set includes the resource features of the preset multimedia resources in multiple channels of the task model to be processed, which are output by the task model to be processed after performing task processing on the preset multimedia resources.

[0160] The feature set partitioning module 820 is configured to partition the current processed feature set into a first channel feature and a current channel feature set; the first channel feature is a resource feature in a channel whose feature modulus information satisfies a preset modulus condition.

[0161] The feature comparison module 830 is configured to perform feature comparison on each resource feature in the first channel feature set and the current channel feature set to obtain the feature comparison result.

[0162] The feature deletion module 840 is configured to delete the second channel feature from the current channel feature set if the feature comparison result indicates that the current channel feature set contains a second channel feature.

[0163] The current processing feature set update module 850 is configured to use the current channel feature set after deleting the second channel feature set as the current processing feature set.

[0164] The loop processing module 860 is configured to repeatedly perform the steps of dividing the current processing feature set into the first channel feature and the current channel feature set, and then using the current channel feature set after deleting the second channel feature as the current processing feature set, until the current processing feature set is empty, and to combine the first channel features determined each time into the target channel feature set.

[0165] The pruning module 870 is configured to perform pruning processing on the model of the task to be processed based on the target channel feature set, so as to obtain the target task processing model.

[0166] As an optional embodiment, the feature set partitioning module includes:

[0167] The feature modulus determination unit is configured to determine the feature modulus information of each resource feature in the current processing feature set.

[0168] The target modulus determination unit is configured to determine the maximum value in the feature modulus information as the target modulus information;

[0169] The first channel feature determination unit is configured to use the resource features corresponding to the target module length information as the first channel features;

[0170] The current channel feature set determination unit is configured to use the resource features other than the first channel features in the current processing feature set as the current channel feature set.

[0171] As an optional embodiment, the feature modulus determination unit includes:

[0172] The normalization processing unit is configured to perform feature normalization processing on each resource feature in the current processing feature set to obtain feature modulus information.

[0173] As an optional embodiment, the feature comparison module includes:

[0174] The feature similarity determination unit is configured to determine the feature similarity between the first channel feature and each resource feature in the current channel feature set;

[0175] The similarity comparison unit is configured to perform a comparison between feature similarity and a similarity threshold to obtain the feature comparison result.

[0176] As an optional embodiment, the apparatus further includes:

[0177] The initial resource feature acquisition module is configured to acquire the initial resource features corresponding to each channel in the network layer of the model to be processed.

[0178] The feature pooling module is configured to perform feature pooling processing on the initial resource features to obtain the resource pooled features corresponding to each channel; and use the resource pooled features corresponding to each channel as the initial channel feature set.

[0179] As an optional embodiment, the pruning processing module includes:

[0180] The target channel determination unit is configured to determine the target channel corresponding to each resource feature in the target channel feature set in the task to be processed model.

[0181] The target task processing model determination unit is configured to perform pruning processing on the channels in the task model to be processed based on the target channel, so as to obtain the target task processing model.

[0182] Figure 9 This is a block diagram of a task processing apparatus according to an exemplary embodiment. (Refer to...) Figure 9 The device includes:

[0183] The pending resource acquisition module 910 is configured to acquire pending multimedia resources.

[0184] The task processing module 920 is configured to process the multimedia resources to be processed by inputting them into the target task processing model generated according to the task processing model generation method described above, and obtain the task processing result.

[0185] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0186] Figure 10 This is a block diagram illustrating an electronic device for task processing model generation or task processing according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 10 As shown, the electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a task processing model generation or task processing method.

[0187] Those skilled in the art will understand that Figure 10The structure shown is merely a block diagram of a portion of the structure related to the present disclosure and does not constitute a limitation on the electronic device to which the present disclosure is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0188] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement a task processing model generation or task processing method as described in the embodiments of this disclosure.

[0189] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of an electronic device to perform the above-described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0190] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the task processing model generation or task processing method of the embodiments of this disclosure.

[0191] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0192] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0193] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for generating a task processing model, characterized in that, The method includes: The initial channel feature set output by the task model to be processed is used as the current processing feature set; the initial channel feature set includes the resource features of the preset multimedia resources in multiple channels of the task model to be processed, which are output by the task model to be processed after performing task processing on the preset multimedia resources. The current processing feature set is divided into a first channel feature set and a current channel feature set; the first channel feature set consists of resource features in the channel whose feature modulus information satisfies a preset modulus condition. The feature comparison is performed on each resource feature in the first channel feature set and the current channel feature set to obtain the feature comparison result; If the feature comparison result indicates that a second channel feature exists in the current channel feature set, the second channel feature is deleted from the current channel feature set; the second channel feature is any channel feature in the current channel feature set whose feature similarity with the first channel feature meets a preset similarity condition; The current channel feature set after deleting the second channel feature is taken as the current processing feature set; Repeat the steps from dividing the current processing feature set into the first channel feature set and the current channel feature set to using the current channel feature set after deleting the second channel feature set as the current processing feature set, until the current processing feature set is an empty set, and combine the first channel features determined each time to form the target channel feature set; Based on the target channel feature set, the task model to be processed is pruned to obtain the target task processing model.

2. The task processing model generation method according to claim 1, characterized in that, The step of dividing the current processed feature set into a first channel feature set and a current channel feature set includes: Determine the feature modulus information of each resource feature in the current processing feature set; The maximum value in the feature modulus information is determined as the target modulus information; The resource features corresponding to the target module length information are used as the first channel features; The current channel feature set is composed of the resource features other than the first channel feature in the current processing feature set.

3. The task processing model generation method according to claim 2, characterized in that, The step of determining the feature modulus information of each resource feature in the current processing feature set includes: For each resource feature in the current processing feature set, feature normalization processing is performed to obtain the feature modulus information.

4. The task processing model generation method according to claim 1, characterized in that, The preset similarity condition is that the feature similarity is greater than or equal to a preset similarity threshold. The feature comparison between the first channel feature and the current channel feature set to obtain the feature comparison result includes: Determine the feature similarity between the first channel feature and each resource feature in the current channel feature set; The feature similarity is compared with the similarity threshold to obtain the feature comparison result.

5. The task processing model generation method according to any one of claims 1 to 4, characterized in that, Before using the initial channel feature set output by the task model as the current processing feature set, the method further includes: Obtain the initial resource features corresponding to each channel in the network layer of the task model to be processed; The initial resource features are subjected to feature pooling to obtain the resource pooling features corresponding to each channel; The resource pooling features corresponding to each channel are combined to form the initial channel feature set.

6. The task processing model generation method according to claim 5, characterized in that, The step of pruning the task model to be processed based on the target channel feature set to obtain the target task processing model includes: In the task-to-be-processed model, determine the target channel corresponding to each resource feature in the target channel feature set; Based on the target channel, the channels in the task-to-be-processed model are pruned to obtain the target task processing model.

7. A task processing method, characterized in that, The method includes: Acquire multimedia resources to be processed; The multimedia resources to be processed are input into the target task processing model generated by the task processing model generation method according to any one of claims 1 to 6 for task processing, and the task processing result is obtained.

8. A task processing model generation device, characterized in that, The device includes: The current processing feature set determination module is configured to use the initial channel feature set output by the task model to be processed as the current processing feature set; the initial channel feature set includes the resource features of the preset multimedia resources in multiple channels of the task model to be processed, which are output by the task model to be processed after performing task processing on the preset multimedia resources. The feature set partitioning module is configured to partition the current processed feature set into a first channel feature and a current channel feature set; the first channel feature is a resource feature in a channel whose feature modulus information satisfies a preset modulus condition. The feature comparison module is configured to perform feature comparison on each resource feature in the first channel feature set and the current channel feature set to obtain feature comparison results; The feature deletion module is configured to delete the second channel feature from the current channel feature set when the feature comparison result indicates that the current channel feature set contains a second channel feature. The current processing feature set update module is configured to use the current channel feature set after deleting the second channel feature as the current processing feature set; The loop processing module is configured to repeatedly perform the steps of dividing the current processing feature set into a first channel feature and a current channel feature set to using the current channel feature set after deleting the second channel feature as the current processing feature set, until the current processing feature set is an empty set, and to combine the first channel features determined each time into a target channel feature set; The pruning module is configured to perform pruning on the task model to be processed based on the target channel feature set, so as to obtain the target task processing model.

9. A task processing device, characterized in that, The device includes: The pending resource acquisition module is configured to acquire pending multimedia resources; The task processing module is configured to input the multimedia resources to be processed into a target task processing model generated by the task processing model generation method according to any one of claims 1 to 6, and to obtain the task processing result.

10. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the task processing model generation method as described in any one of claims 1 to 6 or the task processing method as described in claim 7.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the task processing model generation method as described in any one of claims 1 to 6 or the task processing method as described in claim 7.

Images