Processing method and device of model and vehicle

Abstract

This invention discloses a model processing method, apparatus, and vehicle. The method includes: inputting first training data into a first model to obtain first output values ​​for multiple channels in the first model; constructing a second loss function for each channel based on the first output values ​​and first variables of the multiple channels; training the first model based on the second loss function and second training data for each channel to obtain a first training result; and pruning multiple channels of the first model based on the first training result to obtain a target model. This invention solves the technical problem of low accuracy during model pruning in related technologies.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a method, apparatus, and vehicle for processing models. Background Technology

[0002] Generally, when building neural network models, such as visual perception models, redundant neuron connections may occur, causing the entire model to become overly complex and affecting its running efficiency. In such cases, lightweight pruning is needed to improve the model's efficiency. However, current model pruning methods suffer from problems such as inaccurate differentiation of component channel parameters and excessively fine pruning granularity, resulting in poor accuracy during model pruning.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a model processing method, apparatus, and vehicle to at least solve the technical problem of low accuracy when performing model pruning in related technologies.

[0005] According to one aspect of the present invention, a method for processing a model is provided, comprising: inputting first training data into a first model to obtain first output values ​​of multiple channels in the first model, wherein the first model is used to process a visual perception task of video or image; constructing a second loss function for each channel based on the first output values ​​of the multiple channels and first variables of the multiple channels, wherein the first variables are used to represent the importance of the channel; training the first model based on the second loss function of each channel and second training data to obtain a first training result, wherein the first training result is used to represent the importance of each channel in the first model; pruning multiple channels of the first model based on the first training result to obtain a target model, wherein the target model is used to process a visual perception task of video or image, and the processing speed of the target model for the visual perception task is greater than the processing speed of the first model for the visual perception task.

[0006] Optionally, pruning multiple channels of the first model based on the first training result to obtain the target model includes: sorting multiple channels based on the first training result to obtain a sorting result; determining the first channel among multiple channels based on the sorting result, wherein the target value of the first variable corresponding to the first channel is less than a preset threshold; and deleting the first channel in the first model to obtain the target model.

[0007] Optionally, after sorting multiple channels based on the first training result to obtain the sorting result, the method further includes: determining a preset number of second channels among the multiple channels based on the sorting result; deleting the second channels in the first model to obtain the target model.

[0008] Optionally, pruning multiple channels of the first model based on the first training result to obtain the target model includes: pruning multiple channels of the first model based on the first training result to obtain the second model; and training the second model based on the third training data to obtain the target model.

[0009] Optionally, the target model is trained based on the second loss function of each channel and the second training data to obtain a first training result, including: obtaining the first loss function of the first model, wherein the first loss function is used to represent the loss function corresponding to the second variable, and the second variable is used to represent the accuracy of the first model; obtaining the first product of the second loss function and the first preset value; determining the target loss function based on the sum of the first product and the first loss function; and training the first model based on the target loss function and the second training data to obtain the first training result.

[0010] Optionally, based on the first output values ​​of multiple channels and the first variables of multiple channels, a second loss function is constructed for each channel, including: determining the augmented data values ​​corresponding to the first output values ​​of multiple channels; and determining the second loss function for each channel based on the product of the augmented data values ​​and the first variables of multiple channels.

[0011] Optionally, the method further includes: obtaining the convolutional layer parameters and batch normalization layer parameters of the initial model, wherein the initial model is used to process visual perception tasks of videos or images; fusing the convolutional layer parameters and batch normalization layer parameters to obtain fused parameters; replacing the convolutional layer parameters and batch normalization parameters in the initial model with the fused parameters to obtain a first model, wherein the processing speed of the first model for visual perception tasks is greater than that of the initial model for visual perception tasks.

[0012] According to another aspect of the present invention, a model processing apparatus is also provided, comprising: an acquisition module, configured to input first training data into a first model to obtain first output values ​​of multiple channels in the first model, wherein the first model is used to process a visual perception task of video or image; a construction module, configured to construct a second loss function for each channel based on the first output values ​​of multiple channels and first variables of multiple channels, wherein the first variables are used to represent the importance of the channel; a training module, configured to train the first model based on the second loss function of each channel and second training data to obtain a first training result, wherein the first training result is used to represent the importance of each channel in the first model; and a pruning module, configured to prune multiple channels of the first model based on the first training result to obtain a target model, wherein the target model is used to process a visual perception task of video or image, and the processing speed of the target model for the visual perception task is greater than that of the first model for the visual perception task.

[0013] Optionally, the pruning module includes: a channel sorting unit, used to sort multiple channels based on the first training result to obtain a sorting result; a first channel determination unit, used to determine the first channel among multiple channels based on the sorting result, wherein the target value of the first variable corresponding to the first channel is less than a preset threshold; and a first channel deletion unit, used to delete the first channel in the first model to obtain the target model.

[0014] Optionally, the pruning module further includes: a second channel determination unit, used to determine a preset number of second channels among multiple channels based on the sorting results; and a second channel deletion unit, used to delete the second channels in the first model to obtain the target model.

[0015] Optionally, the pruning module further includes: a channel pruning unit, used to prune multiple channels in the first model based on the first training result to obtain a second model; and a model training unit, used to train the second model based on the third training data to obtain a target model.

[0016] Optionally, the training module includes: a first acquisition unit for acquiring a first loss function of the first model, wherein the first loss function represents the loss function corresponding to the second variable, and the second variable represents the accuracy of the first model; a second acquisition unit for acquiring a first product of the second loss function and a first preset value; a target loss function determination unit for determining a target loss function based on the sum of the first product and the first loss function; and a training result acquisition unit for training the first model based on the target loss function and the second training data to obtain a first training result.

[0017] Optionally, the construction module includes: a data value determination unit for determining the augmented data values ​​corresponding to the first output values ​​of multiple channels; and a second loss function determination unit for determining the second loss function for each channel based on the augmented data values ​​and the product of the first variables of multiple channels.

[0018] Optionally, the device further includes: a first acquisition module for acquiring convolutional layer parameters and batch normalization layer parameters of an initial model, wherein the initial model is used to process visual perception tasks of videos or images; a parameter fusion module for fusing the convolutional layer parameters and batch normalization layer parameters to obtain fused parameters; and a parameter replacement module for replacing the convolutional layer parameters and batch normalization parameters in the initial model with the fused parameters to obtain a first model, wherein the processing speed of the first model for visual perception tasks is greater than that of the initial model for visual perception tasks.

[0019] According to another aspect of the present invention, a computer-readable storage medium is also provided, including a stored program, wherein, when the program is running, it controls the device where the computer-readable storage medium is located to execute the processing method of any of the above models.

[0020] According to another aspect of the present invention, a processor is also provided, wherein the processor's program execution performs the processing method of any of the above-described models.

[0021] According to another aspect of the present invention, a target vehicle is also provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to perform the processing method of any of the above models.

[0022] In this embodiment of the invention, the first output value of the first model is first determined using the first training data. Based on this first output value and the first variable corresponding to each channel in the first model, a second loss function is determined. Then, the first model is trained a second time using the second loss function and the second training data to obtain the first training result. Finally, the model is pruned based on the first training result corresponding to each channel. By determining the L2 norm corresponding to each channel in the first model, i.e., the aforementioned first output value, overfitting during model pruning can be prevented, improving the model's generalization ability. Simultaneously, the L2 norm is used to construct the second loss function to determine the first training result, thereby filtering out network channels with less impact on the first model. Using unified filtering of all network channels instead of individual filtering effectively avoids pruning errors caused by the inability to effectively identify individual network channels, thus solving the technical problem of low accuracy in model pruning in related technologies. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0024] Figure 1 This is a flowchart illustrating a model processing method according to an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of a single-layer structure of a visual perception model according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram illustrating a model pruning training process according to an embodiment of the present invention;

[0027] Figure 4 This is a structural block diagram of a model processing device according to an embodiment of the present invention. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0030] Currently, model pruning methods can generally be divided into two categories: structured pruning and unstructured pruning. Unstructured pruning mainly achieves this by directly trimming model weights, such as resetting the weights of convolutional layers in the model. However, this method has a fine granularity, which may result in overly sparse weights after pruning. Since most hardware cannot support sparse convolution acceleration, unstructured pruning cannot significantly improve speed on embedded chips, thus having significant limitations. Structured pruning methods mainly achieve this by trimming network channels in the model. A channel score threshold can be used to filter and remove network channels with low scores, such as by using cross-entropy, clustering, and other methods to score network channels. However, this method is closely related to the number of channels. When there are many channels, it cannot effectively distinguish channel parameters located in the middle distribution, which leads to inaccurate final channel scores and may result in pruning errors.

[0031] Example 1

[0032] Considering that structured pruning does not require a special database to accelerate inference and has wide applicability, this application proposes a structured pruning method. According to an embodiment of the present invention, a model processing method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0033] Figure 1 This is a flowchart illustrating a model processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0034] Step S102: Input the first training data into the first model to obtain the first output values ​​of multiple channels in the first model.

[0035] The first model is used to handle visual perception tasks of videos or images.

[0036] The first model mentioned above can refer to a visual perception model used to process videos or images, which may include, but is not limited to: VGG (a type of convolutional neural network) model, YOLO (a type of convolutional neural network) model, etc. In addition to visual perception network models, it can also be some common neural network models, without specific limitations here.

[0037] The aforementioned first training data can refer to any data used for visual training, or it can be the data used to train the accuracy of the first model in the previous training, such as images, videos, etc., without any specific limitation here.

[0038] During the model pruning process, the score corresponding to each channel in the model can be calculated first. That is, the first output value of each channel can be determined by using the first training data and the channel parameters corresponding to multiple channels of the first model.

[0039] In one optional embodiment, the L2 norm can be used as the first output value to prevent overfitting during model pruning and improve the model's generalization ability. Furthermore, since the L2 norm is a squared value, using it as the first output value makes the different importance levels between channels in the first model more apparent, thereby improving the efficiency of determining the channels that need pruning. In other words, the formula for calculating the first output value can be:

[0040] ,

[0041] Among them, the above-mentioned This represents the first output value. The 'm' above represents the number of parameters in each network channel of the first model, and the 'j' above represents the j-th parameter out of those m parameters. This represents the output value corresponding to the j-th parameter in the network channel.

[0042] Step S104: Based on the first output values ​​of multiple channels and the first variables of multiple channels, construct the second loss function for each channel.

[0043] The first variable represents the importance of the channel.

[0044] The first variable mentioned above can be a trainable factor s. In an optional scheme of this embodiment, the trainable factor s of each channel can be different. The trainable factor can be set according to the actual situation without specific limitation. Taking the visual perception model as an example, the importance mentioned above can refer to the degree of role played by each channel in a visual perception process, such as the number of calculations of the channel, the impact of the result calculated by the channel on the overall perception result, etc.

[0045] Optionally, based on the first output values ​​of multiple channels and the first variables of multiple channels, a second loss function is constructed for each channel, including: determining the augmented data values ​​corresponding to the first output values ​​of multiple channels; and determining the second loss function for each channel based on the product of the augmented data values ​​and the first variables of multiple channels.

[0046] The aforementioned enhanced data value can refer to a value used to improve the recognition of the first output value of each channel. If the first output value itself is large, the aforementioned enhanced data value can be 1.

[0047] In one optional embodiment, the second loss function can be the product of the first variable and the augmented data value; that is, the formula for calculating the second loss function can be:

[0048] ,

[0049] Among them, the above-mentioned This represents the second loss function, and s above represents the trainable factor, i.e., the first variable.

[0050] Step S106: Train the first model based on the second loss function and the second training data for each channel to obtain the first training result.

[0051] The first training result is used to represent the importance of each channel in the first model.

[0052] The second training data mentioned above can refer to the data used to train the first model. It can be the same as the first training data mentioned above, or different data can be set based on the actual situation. No specific limitation is made here.

[0053] After determining the second loss function, the first model can be retrained using the second training data to obtain the first training result, as shown below:

[0054] Optionally, the target model is trained based on the second loss function of each channel and the second training data to obtain a first training result, including: obtaining the first loss function of the first model, wherein the first loss function is used to represent the loss function corresponding to the second variable, and the second variable is used to represent the accuracy of the first model; obtaining the first product of the second loss function and the first preset value; determining the target loss function based on the sum of the first product and the first loss function; and training the first model based on the target loss function and the second training data to obtain the first training result.

[0055] The aforementioned first loss function can refer to the original network loss function in the first model. This network loss function can be determined according to the specific type and formula of the first model. For details, please refer to relevant literature. It will not be elaborated here.

[0056] The first preset value mentioned above can be the loss function coefficient corresponding to the aforementioned L2 norm. Generally, the first preset value can be used to adjust the preset value of the ratio between the first loss function and the second loss function in order to improve the accuracy of the determined first training result.

[0057] In one optional embodiment, the target loss function can be determined based on the first preset value, the second loss function, and the original first loss function of the first model. The corresponding calculation formula can be:

[0058]

[0059] Among them, the above-mentioned This represents the target loss function, as mentioned above. This represents the first loss function, as mentioned above. This represents the first preset value, as mentioned above. This represents the second loss function.

[0060] In one optional embodiment, the sum of the products of the second loss function and the first preset value corresponding to all channels can be determined first, and then the original first loss function in the first model can be introduced on this basis to make the target loss function determined more accurate.

[0061] After determining the target loss function, the first model can be further trained based on the target loss function and the second training data mentioned above to obtain the scaling factor corresponding to different channels in each layer of the first model, i.e. the first training result mentioned above, and the channels that need to be pruned can be determined based on the first training result.

[0062] Step S108: Based on the first training result, prune multiple channels of the first model to obtain the target model.

[0063] The target model is used to process visual perception tasks of videos or images, and the target model processes visual perception tasks faster than the first model.

[0064] The target model mentioned above can refer to the model after pruning, which processes data faster and is more efficient than the first model before pruning.

[0065] In one optional embodiment, it can be determined whether to prune a channel based on the size of the first training result corresponding to each channel. The specific pruning steps are as follows:

[0066] Optionally, pruning multiple channels of the first model based on the first training result to obtain the target model includes: sorting multiple channels based on the first training result to obtain a sorting result; determining the first channel among multiple channels based on the sorting result, wherein the target value of the first variable corresponding to the first channel is less than a preset threshold; and deleting the first channel in the first model to obtain the target model.

[0067] In one optional embodiment, the first model can be pruned based on the determined size of the first training result for each channel. To improve the efficiency of the model pruning process, the first training results can be arranged in ascending order, and a numerical threshold can be set. All channels smaller than the numerical threshold are identified as the first channels. Since the first training result corresponding to each channel can represent its importance in the first model, if the first training result is smaller than the numerical threshold, it indicates that the channel has low importance in the first model, and the channel can be deleted to achieve the purpose of model pruning.

[0068] Optionally, after sorting multiple channels based on the first training result to obtain the sorting result, the method further includes: determining a preset number of second channels among the multiple channels based on the sorting result; deleting the second channels in the first model to obtain the target model.

[0069] In one optional embodiment, in addition to pruning the first model based on a numerical threshold as described above, pruning can also be performed based on the proportion of the number of first training results. Since some embedded chips have requirements on the number of channels in the model, the model can be pruned according to the channel limit imposed by the embedded chip. That is, a quantity threshold or proportion threshold can be preset, and channels corresponding to the first training results in the sorted first training results that are less than the quantity threshold or proportion threshold are identified as the first channels and deleted, thereby achieving the purpose of model pruning.

[0070] In this embodiment of the invention, the first output value of the first model is first determined using the first training data. Based on this first output value and the first variable corresponding to each channel in the first model, a second loss function is determined. Then, the first model is trained a second time using the second loss function and the second training data to obtain the first training result. Finally, the model is pruned based on the first training result corresponding to each channel. By determining the L2 norm corresponding to each channel in the first model, i.e., the aforementioned first output value, overfitting during model pruning can be prevented, improving the model's generalization ability. Simultaneously, the L2 norm is used to construct the second loss function to determine the first training result, thereby filtering out network channels with less impact on the first model. Using unified filtering of all network channels instead of individual filtering effectively avoids pruning errors caused by the inability to effectively identify individual network channels, thus solving the technical problem of low accuracy in model pruning in related technologies.

[0071] Optionally, pruning multiple channels of the first model based on the first training result to obtain the target model includes: pruning multiple channels of the first model based on the first training result to obtain the second model; and training the second model based on the third training data to obtain the target model.

[0072] The third training data mentioned above can be the same as the first and second training data mentioned above, or it can be set to other data different from the first and second training data according to actual needs. In an optional scheme of this embodiment, after the initial pruning of multiple channels of the first model is completed, the third training data can be used to perform network transfer learning and model verification on the pruned first model, that is, the second model mentioned above, that is, to retrain the second model and determine whether the final determined model meets the expected requirements.

[0073] For example, if the first model mentioned above is a visual perception model, the pruned model can be retrained using the third training data, and it can be determined whether the pruned first model can accurately identify the model object, such as face recognition or pedestrian recognition. If the final recognition result is relatively accurate and meets the expected requirements, it can be determined that the model pruning effect of the first model is good, and the model trained with the third training data can be determined as the target model.

[0074] Optionally, the processing method of the model in this embodiment further includes: obtaining the convolutional layer parameters and batch normalization layer parameters of the initial model, wherein the initial model is used to process visual perception tasks of videos or images; fusing the convolutional layer parameters and batch normalization layer parameters to obtain fused parameters; replacing the convolutional layer parameters and batch normalization parameters in the initial model with the fused parameters to obtain a first model, wherein the processing speed of the first model for visual perception tasks is greater than that of the initial model for visual perception tasks.

[0075] To improve the efficiency of identifying the position of each channel in the model during pruning, the parameters of the initial model that needs to be pruned can be merged first, that is, the initial model can be optimized first. The optimized initial model is the first model mentioned above.

[0076] Figure 2 This is a schematic diagram of a single-layer structure of a visual perception model according to an embodiment of the present invention, as shown below. Figure 2 As shown, each layer of a visual perception model generally consists of three parts: a convolutional layer, a batch normalization layer, and an activation function layer.

[0077] Specifically, you can first obtain the convolutional layer parameters in the initial model. and batch normalization layer parameters and Among them, the convolutional layer parameters The weights of the initial model in this layer Bias value , and input The parameters obtained in the batch normalization layer Representative convolutional layer parameters mean, parameters This represents parameters. The variance.

[0078] In one optional embodiment, the above convolutional layer parameters The calculation formula is as follows:

[0079] ,

[0080] mean The calculation formula is as follows:

[0081] ,

[0082] variance The calculation formula is as follows:

[0083] ,

[0084] The 'm' mentioned above represents a total of m channels.

[0085] After determining the parameters of the convolutional layers and the batch normalization layers mentioned above, they can be fused to simplify the model pruning process and improve the efficiency of model pruning.

[0086] In one optional embodiment, the fusion formula may be:

[0087] ,

[0088] Among them, the above-mentioned This represents the aforementioned fusion parameters, specifically the output value corresponding to each channel parameter mentioned above. and These represent the learnable parameters in the model network, which are generally used to recover the details of the normalized model features, as mentioned above. This represents a local minimum value, which is generally used to ensure the stability of the above formula and prevent the aforementioned variance. A value of 0 is generally acceptable. For e -6 .

[0089] After determining the above fusion parameters, Bina can use these fusion parameters to directly replace the convolutional layer parameters and batch normalization layer parameters, thereby improving the running efficiency of the first model.

[0090] To facilitate understanding of the above model processing method, Figure 3 This is a schematic diagram illustrating a model pruning training process according to an embodiment of the present invention, as shown below. Figure 3As shown, the convolutional layer parameters and batch normalization layer parameters in the initial model are first fused to obtain fused parameters, which are then used to replace the original convolutional layer parameters and batch normalization layer parameters to obtain the first model. Next, the L2 norm value of each channel output in the first model is calculated. Using the L2 norm and the trainable factor, the corresponding second loss function is constructed, and combined with the original first loss function of the model, the target loss function is determined. Then, using the target loss function and training data, the first training result of each channel is determined, and multiple channels in the model are pruned according to a preset threshold to obtain the target model after model pruning. Finally, the target model is trained using new training data to complete the entire model pruning process.

[0091] Example 2

[0092] According to another aspect of the embodiments of the present invention, this specification also provides a model processing apparatus, please refer to Figure 4 , Figure 4 This is a structural block diagram of a model processing device according to an embodiment of the present invention. The device includes: an acquisition module 402, used to input first training data into a first model to obtain first output values ​​of multiple channels in the first model, wherein the first model is used to process visual perception tasks of videos or images; a construction module 404, used to construct a second loss function for each channel based on the first output values ​​of multiple channels and first variables of multiple channels, wherein the first variables are used to represent the importance of the channel; a training module 406, used to train the first model based on the second loss function of each channel and second training data to obtain a first training result, wherein the first training result is used to represent the importance of each channel in the first model; and a pruning module 408, used to prune multiple channels of the first model based on the first training result to obtain a target model, wherein the target model is used to process visual perception tasks of videos or images, and the processing speed of the target model for visual perception tasks is greater than that of the first model for visual perception tasks.

[0093] Optionally, the pruning module 408 includes: a channel sorting unit, used to sort multiple channels based on the first training result to obtain a sorting result; a first channel determination unit, used to determine the first channel among multiple channels based on the sorting result, wherein the target value of the first variable corresponding to the first channel is less than a preset threshold; and a first channel deletion unit, used to delete the first channel in the first model to obtain the target model.

[0094] Optionally, the pruning module 408 further includes: a second channel determination unit, used to determine a preset number of second channels among multiple channels based on the sorting results; and a second channel deletion unit, used to delete the second channels in the first model to obtain the target model.

[0095] Optionally, the pruning module 408 further includes: a channel pruning unit, used to prune multiple channels in the first model based on the first training result to obtain a second model; and a model training unit, used to train the second model based on the third training data to obtain a target model.

[0096] Optionally, the training module 406 includes: a first acquisition unit for acquiring a first loss function of the first model, wherein the first loss function represents the loss function corresponding to the second variable, and the second variable represents the accuracy of the first model; a second acquisition unit for acquiring a first product of the second loss function and a first preset value; a target loss function determination unit for determining a target loss function based on the sum of the first product and the first loss function; and a training result acquisition unit for training the first model based on the target loss function and the second training data to obtain a first training result.

[0097] Optionally, the construction module 404 includes: a data value determination unit, used to determine the enhanced data values ​​corresponding to the first output values ​​of multiple channels; and a second loss function determination unit, used to determine the second loss function of each channel based on the product of the enhanced data values ​​and the first variables of multiple channels.

[0098] Optionally, the device further includes: a first acquisition module for acquiring convolutional layer parameters and batch normalization layer parameters of an initial model, wherein the initial model is used to process visual perception tasks of videos or images; a parameter fusion module for fusing the convolutional layer parameters and batch normalization layer parameters to obtain fused parameters; and a parameter replacement module for replacing the convolutional layer parameters and batch normalization parameters in the initial model with the fused parameters to obtain a first model, wherein the processing speed of the first model for visual perception tasks is greater than that of the initial model for visual perception tasks.

[0099] Example 3

[0100] According to another aspect of the present invention, a computer-readable storage medium is also provided, including a stored program, wherein, when the program is running, it controls the device where the computer-readable storage medium is located to execute the processing method of any of the above models.

[0101] Example 4

[0102] According to another aspect of the present invention, a processor is also provided, wherein the processor's program execution performs the processing method of any of the above-described models.

[0103] Example 5

[0104] According to another aspect of the present invention, a target vehicle is also provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to perform the processing method of any of the above models.

[0105] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0106] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0107] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0108] The units described as separate components may or may not be physically separate. Similarly, the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0110] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0111] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for processing a model, characterized in that, include: The first training data is input into the first model to obtain the first output value of multiple channels in the first model, wherein the first model is used to process visual perception tasks of videos or images. Based on the first output values ​​of the multiple channels and the first variables of the multiple channels, a second loss function is constructed for each channel, wherein the first variables are used to represent the importance of the channel; Obtain the first loss function of the first model, wherein the first loss function is used to represent the loss function corresponding to the second variable, and the second variable is used to represent the accuracy of the first model; Obtain the first product of the second loss function and the first preset value for each channel; Based on the sum of the first product and the first loss function, the target loss function is determined; The first model is trained based on the target loss function and the second training data to obtain a first training result, wherein the first training result is used to represent the importance of each channel in the first model; Based on the first training result, the multiple channels in the first model are pruned to obtain the second model; The second model is validated based on the third training data to obtain a target model, wherein the target model is used to process the visual perception task of the video or the image, and the processing speed of the target model for the visual perception task is greater than that of the first model for the visual perception task.

2. The method according to claim 1, characterized in that, Based on the first training result, the multiple channels of the first model are pruned to obtain a second model, including: Based on the first training result, the multiple channels are sorted to obtain the sorting result; Based on the sorting result, a first channel is determined among the multiple channels, wherein the target value of the first variable corresponding to the first channel is less than a preset threshold. Delete the first channel from the first model to obtain the second model.

3. The method according to claim 2, characterized in that, After sorting the multiple channels based on the first training result to obtain the sorting result, the method further includes: Based on the sorting results, a preset number of second channels are determined among the plurality of channels; Delete the second channel from the first model to obtain the second model.

4. The method according to claim 1, characterized in that, Based on the first output values ​​of the multiple channels and the first variables of the multiple channels, a second loss function is constructed for each channel, including: Determine the enhanced data value corresponding to the first output value of the plurality of channels, wherein the enhanced data value refers to the value used to improve the recognition of the first output value of each channel; The second loss function for each channel is determined based on the product of the enhanced data value and the first variable of the plurality of channels.

5. The method according to claim 1, characterized in that, The method further includes: Obtain the convolutional layer parameters and batch normalization layer parameters of the initial model, wherein the initial model is used to process the visual perception task of the video or image; The convolutional layer parameters and the batch normalization layer parameters are fused to obtain fused parameters; The convolutional layer parameters and the batch normalization layer parameters in the initial model are replaced with the fusion parameters to obtain the first model, wherein the processing speed of the first model for the visual perception task is greater than that of the initial model for the visual perception task.

6. A model processing apparatus, characterized in that, include: The acquisition module is used to input the first training data into the first model and obtain the first output value of multiple channels in the first model, wherein the first model is used to process the visual perception task of video or image. A construction module is used to construct a second loss function for each channel based on the first output value of the plurality of channels and the first variable of the plurality of channels, wherein the first variable is used to represent the importance of the channel; The training module is used to obtain a first loss function of the first model, wherein the first loss function represents the loss function corresponding to the second variable, and the second variable represents the accuracy of the first model; obtain a first product of the second loss function and a first preset value for each channel; determine a target loss function based on the sum of the first product and the first loss function; and train the first model based on the target loss function and the second training data to obtain a first training result, wherein the first training result represents the importance of each channel in the first model. The pruning module is used to prune the multiple channels in the first model based on the first training result to obtain a second model; and to perform model validation on the second model based on the third training data to obtain a target model, wherein the target model is used to process the visual perception task of the video or the image, and the processing speed of the target model for the visual perception task is greater than that of the first model for the visual perception task.

7. A vehicle, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the processing method of the model according to any one of claims 1-5.

8. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program is executed, it controls the processor of the device to execute the processing method of the model according to any one of claims 1-5.

Images