A model training method and device, a storage medium, and an electronic device

Abstract

Embodiments of the present application provide a model training method and device, a storage medium and an electronic device, wherein the method comprises: identifying training data from N target dimensions by using a target teacher model trained, to determine a first feature map of the training data and a first identification result of the training data in each target dimension; identifying the training data from the N target dimensions by using an initial student model, to determine a second feature map of the training data and a second identification result of the training data in each target dimension; determining a target loss value of the initial student model based on the first identification result, the second identification result, the first feature map and the second feature map; and in the case that the target loss value does not satisfy a predetermined condition, updating network parameters of the initial student model until the target loss value satisfies the predetermined condition, to obtain a target network model. Through the present application, a target network model can be used to perform different tasks, and the efficiency of training the model is improved.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more specifically, to a training method, apparatus, storage medium, and electronic device for a model. Background Technology

[0002] Knowledge distillation is a commonly used compression technique that is relatively simple to operate and can achieve good performance. It employs a teacher-student model framework, using a complex and highly accurate model as the teacher model and a simple, lightweight network as the student model. The teacher model has strong learning capabilities, and during training, knowledge from the teacher model is transferred to the less capable student model to enhance its learning and generalization abilities. The goal is for the lightweight student model to learn with a level of accuracy similar to the teacher model, and this lightweight student model is ultimately deployed online.

[0003] Common knowledge distillation methods are mainly used for single-task classification, where each model is only used to perform a single action. When multiple tasks need to be performed, multiple models need to be trained, with each model used to perform one task.

[0004] This indicates that the related technologies suffer from the problem of the model performing only a single task.

[0005] There is currently no effective solution to the aforementioned problems in the relevant technologies. Summary of the Invention

[0006] The present invention provides a model training method, apparatus, storage medium, and electronic device to at least solve the problem of single task execution of models in related technologies.

[0007] According to an embodiment of the present invention, a method for training a model is provided, comprising: identifying training data from N target dimensions using a trained target teacher model, determining a first feature map of the training data and a first identification result of the training data in each target dimension, wherein the target teacher model includes the N first sub-models, and one first sub-model is used to identify the training data from one target dimension; identifying the training data from the N target dimensions using an initial student model, determining a second feature map of the training data and a second identification result of the training data in each target dimension, wherein the initial student model is a network model obtained after initial training, the initial student model includes the N second sub-models, and one second sub-model is used to identify the training data from one target dimension; determining a target loss value of the initial student model based on the first identification result, the second identification result, the first feature map, and the second feature map; and updating the network parameters of the initial student model if the target loss value does not meet a predetermined condition, until the target loss value meets the predetermined condition, thereby obtaining a target network model.

[0008] According to another embodiment of the present invention, a model training apparatus is provided, comprising: a first identification module, configured to identify training data from N target dimensions using a trained target teacher model, and determine a first feature map of the training data and a first identification result of the training data in each target dimension, wherein the target teacher model includes the N first sub-models, and one first sub-model is used to identify the training data from one target dimension; a second identification module, configured to identify the training data from the N target dimensions using an initial student model, and determine a second feature map of the training data and a second identification result of the training data in each target dimension, wherein the initial student model is a network model obtained after initial training, and the initial student model includes the N second sub-models, and one second sub-model is used to identify the training data from one target dimension; a determination module, configured to determine a target loss value of the initial student model based on the first identification result, the second identification result, the first feature map, and the second feature map; and a training module, configured to update the network parameters of the initial student model if the target loss value does not meet a predetermined condition, until the target loss value meets the predetermined condition, thereby obtaining a target network model.

[0009] According to yet another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0010] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0011] This invention utilizes a trained target teacher model to identify training data from N target dimensions, determining the first feature map of the training data and the first identification result of the training data in each target dimension. An initial student model is then used to identify training data from N target dimensions, determining the second feature map of the training data and the second identification result of the training data in each target dimension. Based on the first and second identification results, the first and second feature maps, a target loss value for the initial student model is determined. If the target loss value does not meet predetermined conditions, the network parameters of the initial student model are updated until the target loss value meets the predetermined conditions, thus obtaining the target student model. The target teacher model includes N first sub-models, each used to identify training data from one target dimension. The initial student model includes N second sub-models, each used to identify training data from one target dimension. Since both the target teacher model and the initial student model can identify training data from different dimensions, the trained target network model can identify data from N different target dimensions, enabling a single target network model to perform different tasks. Therefore, this solves the problem of single-task performance in related technologies, achieving the effect of using a single target network model to perform different tasks and improving the efficiency of model training. Attached Figure Description

[0012] Figure 1 This is a hardware structure block diagram of a mobile terminal for a model training method according to an embodiment of the present invention. Figure 2 This is a flowchart of a model training method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the network architecture of the target teacher model and the initial student model according to an exemplary embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the process of determining a first loss value according to an exemplary embodiment of the present invention; Figure 5 This is a structural block diagram of a model training device according to an embodiment of the present invention. Detailed Implementation

[0013] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and examples.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0015] Currently, recognition tasks based on convolutional neural networks have become the mainstream research direction in the field of vision. In practical applications, due to the high cost of hardware for deploying models and the high performance requirements of GPUs, it is often desirable to deploy models with low memory consumption and low latency. Therefore, the demand for lightweight networks during model deployment is becoming increasingly strong. Generally, models with good accuracy are large networks with many parameters, or even ensembles of multiple models. Such models have slow inference speeds, high resource requirements, and are difficult to deploy directly to services. Model compression has become an important step.

[0016] The mainstream model compression techniques are as follows: (1) Structural optimization: By optimizing the design of the network structure, the redundancy and computation of the model are reduced. For example, improvements at the network block level, such as depthwise separable convolution and grouped convolution, reduce the number of parameters and computation while ensuring network performance. (2) Pruning technique: Based on a pre-trained large model, an evaluation criterion for network parameters is designed, and "redundant" parameters are removed based on this. (3) Quantization technique: Typical 32-bit floating-point network parameters are represented with a lower bit width. Network parameters include weights, activation values, gradients, and errors, etc. (4) Knowledge distillation: Knowledge distillation transfers the knowledge of the teacher network to the student network, enabling the student network to achieve similar performance to the teacher network, while also achieving the purpose of model compression.

[0017] Knowledge distillation is a commonly used compression technique. Compared to other compression techniques, it is relatively simple to operate and can achieve good performance. Knowledge distillation employs a teacher-student model framework, using a complex and highly accurate model as the teacher model and a simple, lightweight network as the student model. The teacher model has strong learning capabilities, and during training, knowledge from the teacher model is transferred to the less capable student model to enhance its learning and generalization abilities. The goal is for the lightweight student model to learn with a level of accuracy similar to the teacher model, and this lightweight student model is ultimately deployed online.

[0018] Common knowledge distillation methods are mainly used for single-task classification. The difficulties of multi-task distillation compared to single-task distillation are: (1) the loss balance problem between different tasks. The problems solved by the images of different tasks are also different. Therefore, the difficulty of sample learning is different, and the loss during training will also be different. Therefore, the learning process should adaptively adjust the loss between each task to prevent the network from over-learning a certain task and ignoring other tasks, resulting in a large performance gap between tasks. (2) Multi-task knowledge distillation is more difficult to learn than single-task learning. As there are more tasks, the knowledge that needs to be learned also increases. Therefore, the learning ability of the knowledge distillation framework is required to be higher.

[0019] To address the aforementioned problems in the relevant technologies, the following embodiments are proposed: The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for a model training method according to an embodiment of the present invention. For example... Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0020] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the model training method in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0021] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0022] This embodiment provides a method for training a model. Figure 2 This is a flowchart of a model training method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S202: Using the trained target teacher model, identify training data from N target dimensions, determine the first feature map of the training data and the first identification result of the training data in each target dimension, wherein the target teacher model includes the N first sub-models, and one first sub-model is used to identify the training data from one target dimension; Step S204: Using the initial student model, identify the training data from the N target dimensions, determine the second feature map of the training data and the second identification result of the training data in each target dimension, wherein the initial student model is a network model obtained after initial training, and the initial student model includes the N second sub-models, one of which is used to identify the training data from one target dimension; Step S206: Determine the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map; Step S208: If the target loss value does not meet the predetermined condition, update the network parameters of the initial student model until the target loss value meets the predetermined condition, and obtain the target network model.

[0023] In the above embodiments, the initial teacher model can be trained to obtain a converged target teacher model. The target teacher model can identify training data from different target dimensions, and each target dimension can be considered a task. For example, when the training data is an image containing motor vehicles, the target dimensions can include license plate, vehicle type, and vehicle orientation. That is, when the training data is an image containing motor vehicles, the target teacher model can identify the image from multiple target dimensions to obtain a first recognition result, which includes license plate information, vehicle type information, and vehicle orientation information. The target teacher model includes N first sub-models, each used to identify training data from one target dimension. Different first sub-models correspond to different target dimensions, and N is a positive integer greater than 1, such as 2 or 3. It should be noted that the above value of N is only an example; N can also be 4, 6, 8, etc. When N is 3, the network architecture diagram of the target teacher model and the initial student model can be found in the appendix. Figure 3 ,like Figure 3 As shown, the model architecture includes three branches, namely three first sub-models or second sub-models, which are used to perform different tasks and obtain prediction values ​​for different tasks, namely the first recognition result or the second recognition result.

[0024] In the above embodiments, the initial student model can be a model trained to convergence using training data. The number of second sub-models included in the initial student model is the same as the number of first sub-models included in the target teacher model.

[0025] In the above embodiments, training datasets corresponding to different target dimensions can be obtained. The balance of training datasets across different target dimensions can be achieved by using a random sampling strategy to expand the training datasets to the same level. For example, the target dimension with the largest training data volume can be selected from N target dimensions, and this largest data volume can be used as the target data volume to be expanded for the remaining dimensions. For the remaining tasks requiring data expansion, the following operation can be performed: random repeated sampling based on the existing dataset until the data volume equals the target data volume. After obtaining the training dataset corresponding to each target dimension, the multi-dimensional training dataset can be input into a teacher network with a large number of parameters and complex training until convergence, obtaining a high-accuracy target teacher model. Simultaneously, the same multi-dimensional training dataset can be input into a student network with fewer parameters and simpler training until convergence, obtaining an initial student model with slightly lower accuracy. Here, parameters refer to the weights and biases of the network's convolutional kernels, the weights and biases of the fully connected layers, and the two learnable variables of the BatchNorm layer, etc. Fewer parameters mean fewer layers of convolutional kernels and a simpler network module structure, resulting in fewer parameters.

[0026] In the above embodiments, after obtaining the target teacher model and the initial student model, the target teacher model and the initial student model can be used to identify training data, obtaining a first feature map and a first recognition result output by the target teacher model, and a second feature map and a second recognition result output by the initial student model. The target loss value of the initial student model is then determined based on the first feature map, the second feature map, the first recognition result, and the second recognition result. If the target loss value does not meet a predetermined condition, the network parameters of the initial student model are updated. The training data included in the training dataset is then input again into the target teacher model and the initial student model with updated network parameters to obtain the target loss value of the initial student model with updated network parameters. If the target loss value does not meet the predetermined condition, the network parameters of the initial student model are updated again. This process continues until the target loss value of the initial student model with updated network parameters meets the predetermined condition, at which point the final model including the target teacher model and the initial teacher model is determined as the target network model.

[0027] In the above embodiments, the target loss value can be determined by the number of training iterations. For example, the predetermined condition can be set as a predetermined number of training iterations. If the number of training iterations corresponding to the current target loss value is less than the predetermined number, the target loss value is considered not to meet the predetermined condition. If the number of training iterations corresponding to the current target loss value is greater than or equal to the predetermined number, the target loss value is considered to meet the predetermined condition.

[0028] In the above embodiments, it can also be determined whether the target loss value meets the predetermined conditions by using a loss value threshold. For example, if the target loss value is less than the loss value threshold, it is determined that the target loss value meets the predetermined conditions; if the target loss value is greater than or equal to the loss value threshold, it is determined that the target loss value does not meet the predetermined conditions.

[0029] In the above embodiments, the predetermined conditions may include both a predetermined number of times and a loss value threshold. When the target loss value satisfies either of the two conditions, the target loss value is considered to satisfy the predetermined conditions.

[0030] Optionally, the entity performing the above steps may be a background processor, or other devices with similar processing capabilities, or at least a machine with integrated data processing equipment. The data processing equipment may include, but is not limited to, terminals such as computers and mobile phones.

[0031] This invention utilizes a trained target teacher model to identify training data from N target dimensions, determining the first feature map of the training data and the first identification result of the training data in each target dimension. An initial student model is then used to identify training data from N target dimensions, determining the second feature map of the training data and the second identification result of the training data in each target dimension. Based on the first and second identification results, the first and second feature maps, a target loss value for the initial student model is determined. If the target loss value does not meet predetermined conditions, the network parameters of the initial student model are updated until the target loss value meets the predetermined conditions, thus obtaining the target student model. The target teacher model includes N first sub-models, each used to identify training data from one target dimension. The initial student model includes N second sub-models, each used to identify training data from one target dimension. Since both the target teacher model and the initial student model can identify training data from different dimensions, the trained target network model can identify data from N different target dimensions, enabling a single target network model to perform different tasks. Therefore, this solves the problem of single-task performance in related technologies, achieving the effect of using a single target network model to perform different tasks and improving the efficiency of model training.

[0032] In an exemplary embodiment, determining the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map includes: determining a first loss value based on the first recognition result and the second recognition result; determining a second loss value based on the first feature map and the second feature map; and determining a target loss value based on the first loss value and the second loss value. In this embodiment, when determining the target loss value, the first loss value between the first recognition result and the second recognition result can be determined separately, the second loss value between the first feature map and the second feature map can be determined, and the target loss value can be determined based on the first loss value and the second loss value.

[0033] In the above embodiments, during the distillation process, the outputs of the N first sub-models of the target teacher model can be used as soft labels to replace the hard labels and perform KL divergence loss on the initial student model to determine the first loss value. Here, the soft labels are the first recognition results. The hard labels are the label information included in the training data, that is, the label information pre-assigned to the training data. A schematic diagram of the process for determining the first loss value can be found in the appendix. Figure 4 The second loss value can be determined using the mean squared error (MSE).

[0034] In an exemplary embodiment, determining a first loss value based on the first identification result and the second identification result includes: determining a first sub-identification result for each target dimension included in the first identification result and a second sub-identification result for each target dimension included in the second identification result; determining a first sub-loss value based on the first sub-identification result and the second sub-identification result for each target dimension; determining a target weight corresponding to each target dimension; and determining the first loss value based on each first sub-loss value and each target weight. In this embodiment, when determining the first loss value, the first sub-identification result and the second sub-identification result corresponding to each target dimension can be determined separately, and a first sub-loss value between the first sub-identification result and the second sub-identification result can be determined to obtain N first sub-loss values. Alternatively, the target weight corresponding to each target dimension can be determined, and the target loss value can be determined based on the N first sub-loss values ​​and the N target weights.

[0035] In the above embodiments, the product of the first sub-loss value and its corresponding target weight can be determined to obtain N products, and the sum of the N products is determined as the first loss value. Alternatively, the first loss value can be determined using other methods based on each first sub-loss value and each target weight.

[0036] In an exemplary embodiment, determining the target weight corresponding to each target dimension includes: when the first loss value is determined for the first time, determining a pre-determined initial weight as the target weight for each target dimension; when the first loss value is not determined for the first time, determining the largest sub-loss value among the N first sub-loss values, increasing the first current weight of the target dimension corresponding to the largest sub-loss value according to a first predetermined method, and decreasing the second current weight of the target dimensions corresponding to other sub-loss values, to obtain the target weight for each target dimension, wherein the other sub-loss values ​​are loss values ​​other than the largest sub-loss value among the N first sub-loss values. In this embodiment, in order to balance the differences in loss among the three tasks, adaptive weights can be used to weight the losses of different tasks, and the weighting weights are automatically set according to the current loss value, i.e., the first sub-loss value. When the first loss value is determined for the first time, the pre-determined initial weight can be determined as the target weight of the target dimension, for example, the target weight can be 1 / N. When the first loss value is determined and the target loss value is obtained, if the target loss value does not meet a predetermined condition, the network parameters of the initial student model are updated. After updating the network parameters, when determining the first loss value again, the corresponding weights can be adjusted based on each previously determined first sub-loss value. The process of adjusting the target weights can involve increasing the first current weight corresponding to the largest sub-loss value and decreasing the second current weights corresponding to the other sub-loss values.

[0037] In an exemplary embodiment, determining the first loss value based on each first sub-loss value and each target weight includes: determining the reciprocal of the product of each target weight and a first constant to obtain N first reciprocals; determining the first product of each first sub-loss value and the first reciprocal corresponding to the first sub-loss value to obtain the N first products; determining the Nth root of the product of the N target weights; determining the logarithm of the Nth root with a second constant as the base; and determining the first sum of the N first products and the logarithm as the first loss value. In this embodiment, the reciprocal of the product of the target weight and the first constant can be determined to obtain N first reciprocals, where the first constant can be N. The first product of each first sub-loss value and its corresponding first reciprocal is determined, the Nth root of the product of the N target weights is determined, the logarithm of the Nth root with a second constant as the base is determined, and the first sum of the N first products and the logarithm is determined as the first loss value. The second constant can be 2, e, 10, etc.

[0038] In the above embodiment, when N is 3, the first loss value can be expressed as: ,in, This is the KL loss for the first task, i.e., the first sub-loss value. For the KL loss in the second task, For the KL loss in the third task, 、 、 Defined as learnable target weights in the network, all initialized to 1, and adaptively changing as the network trains. This represents the total KL loss for the three tasks, i.e., the first loss value. Since we aim to minimize the first three terms during network training, the corresponding... 、 、 As the variable grows larger, the fourth term will also grow larger, creating a counterforce against the first three terms to ensure... The variables are within a reasonable range. If the KL loss for a certain task increases, then the corresponding... This will also increase, indicating that the learning weight of this task has increased.

[0039] In the above embodiment, three learnable variables are defined in the network to represent the loss weights of the three tasks. During the network learning process, the KL loss function in Formula 1 is minimized, and the values ​​of the three learnable variables are updated accordingly, which adaptively adjusts the corresponding loss weights. In an exemplary embodiment, determining a second loss value based on the first feature map and the second feature map includes: determining the mean squared error between the first feature map and the second feature map; and determining the mean squared error as the second loss value. In this embodiment, when determining the second loss value, it can be determined based on the mean squared error between the first feature map and the second feature map, and the mean squared error can be determined as the second loss value. The first feature map can be the last layer feature map of the backbone of the target teacher model, and the second feature map can be the last layer feature map of the backbone of the initial student model.

[0040] In the above embodiment, the last layer feature maps of the teacher network backbone and the student network backbone are compressed into two-dimensional matrices and then aligned to obtain two feature maps of the same dimension. The mean square error (MSE) of both is calculated and used as the second loss value. The steps for calculating the mean square error are as follows: If the size of the feature map is a four-dimensional vector B*C*H*W, B represents the batch size of the input data, C represents the number of channels in the feature map, and H and W are the size of the feature map. Since the initial student model and the target teacher model have the same data batch size B when inputting data, let the feature map of the target teacher model be B*C1*H1*W1, and compress its dimension into a two-dimensional vector B*(C1*H1*W1). The initial student model has a size of B*C2*H2*W2, and its dimension is compressed into a two-dimensional vector B*(C2*H2*W2). Next, a dimension alignment operation is performed. The B*(C1*H1*W1) matrix of the target teacher model is multiplied by its own B*(C1*H1*W1) transpose matrix to obtain a two-dimensional vector B*B. The student network also performs the same operation to obtain a two-dimensional vector B*B. The mean squared error of both is then calculated.

[0041] In an exemplary embodiment, determining a target loss value based on the first loss value and the second loss value includes: determining a first weight corresponding to the first loss value and a second weight corresponding to the second loss value; determining a second product of the first loss value and the first weight; determining a third product of the second loss value and the second weight; and determining a second sum of the second product and the third product as the target loss value. In this embodiment, the sum of the multi-task adaptively weighted KL divergence loss (i.e., the first loss value) and the MSE loss (i.e., the second loss value) can be used as the final loss, i.e., the target loss value, to supervise the entire distillation process and enhance the learning ability of the student network. The formula is as follows: It can represent the target loss value. For N tasks, the KL loss function is used, i.e., the first loss value. This represents the mean squared error loss of the feature map, i.e., the second loss value. 、 For fixed weighting parameters, they can be customized during training based on differences in loss, such as... Take 0.6, Let's take 2000. Here, supervision refers to the process of calculating the gradient of the loss function with respect to the parameters in the network during the backpropagation of training. This gradient is then used to update the various parameters in the network, thereby guiding the direction of the network's learning.

[0042] In an exemplary embodiment, before using the initial student model to identify the training data from the N target dimensions, the method further includes: training the initial model using a training dataset to obtain the initial student model, wherein the training dataset includes the N sub-training datasets, the initial model includes the N initial second sub-models, and one sub-training dataset is used to train one initial second sub-model to obtain the second sub-model. In this embodiment, the initial student model may be a model that has been pre-trained using the training dataset. After the initial student model training converges, it is retrained using the first feature map output by the target teacher model and the first recognition result, thereby improving the accuracy of the initial student model. This further improves the accuracy of the target network model.

[0043] In an exemplary embodiment, after obtaining the target network model, the method further includes: using the target network model to identify a target image from the N target dimensions; and outputting the target recognition result of the target image in the N target dimensions. In this embodiment, after obtaining the target network model, it can be applied to various fields. In different fields, the tasks performed by each sub-model of the target network model, i.e., the target dimensions, are different. For example, when applied to the transportation field, the target dimensions can be vehicle type, license plate, vehicle color, vehicle orientation, etc. When applied to the facial recognition field, the target dimensions can be gender, facial features, facial attributes, and object identification information, such as ID card number, etc.

[0044] In the aforementioned embodiment, a single network is used for distillation learning of N tasks. Before inputting the data into the network, class equalization is performed on the datasets of the N tasks to ensure that the data size is at the same level. The N training datasets are input into the same network (backbone), and during training, the parameters of the same backbone are shared by the N tasks. The output is the predicted values ​​for the N tasks. Both the teacher network (target teacher model) and the student network (initial student model) adopt this multi-task framework. During distillation, the teacher network outputs soft labels to supervise the learning of the student network, and adaptive weights are used to balance the losses between different tasks. To enhance the performance of knowledge distillation, the last layer feature map of the teacher backbone and the last layer feature map of the student backbone are extracted to calculate the loss as part of the distillation loss, thus enhancing the learning ability of the network. Because the distillation loss of different tasks can be adaptively balanced during training, it prevents the model from overlearning one task and ignoring others. At the same time, the MSE loss function is calculated using the last layer feature map to increase the performance of distillation, resulting in more knowledge learned and stronger learning ability than simply using the soft label loss function.

[0045] Compared to existing single-task classification techniques, this multi-task distillation method integrates multiple tasks into a single model for training, simplifying repetitive training steps while achieving performance comparable to single-task classification.

[0046] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0047] This embodiment also provides a model training device for implementing the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0048] Figure 5 This is a structural block diagram of a model training device according to an embodiment of the present invention, such as... Figure 5 As shown, the device includes: The first identification module 52 is used to identify training data from N target dimensions using the trained target teacher model, determine the first feature map of the training data and the first identification result of the training data in each target dimension, wherein the target teacher model includes the N first sub-models, and one first sub-model is used to identify the training data from one target dimension. The second identification module 54 is used to identify the training data from the N target dimensions using an initial student model, determine the second feature map of the training data and the second identification result of the training data in each target dimension, wherein the initial student model is a network model obtained after initial training, and the initial student model includes the N second sub-models, and one second sub-model is used to identify the training data from one target dimension; The determining module 56 is configured to determine the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map; Training module 58 is used to update the network parameters of the initial student model when the target loss value does not meet the predetermined conditions, until the target loss value meets the predetermined conditions, so as to obtain the target network model.

[0049] In an exemplary embodiment, the determining module 56 may determine the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map in the following manner: determining a first loss value based on the first recognition result and the second recognition result; determining a second loss value based on the first feature map and the second feature map; and determining a target loss value based on the first loss value and the second loss value.

[0050] In an exemplary embodiment, the determining module 56 may determine a first loss value based on the first identification result and the second identification result in the following manner: determining a first sub-identification result for each target dimension included in the first identification result and a second sub-identification result for each target dimension included in the second identification result; determining a first sub-loss value based on the first sub-identification result and the second sub-identification result for each target dimension; determining a target weight corresponding to each target dimension; and determining the first loss value based on each first sub-loss value and each target weight.

[0051] In an exemplary embodiment, the determining module 56 can determine the target weight corresponding to each target dimension in the following manner: when the first loss value is determined for the first time, a predetermined initial weight is determined as the target weight of each target dimension; when the first loss value is not determined for the first time, the largest sub-loss value included among the N first sub-loss values ​​is determined, and the first current weight of the target dimension corresponding to the largest sub-loss value is increased according to a first predetermined method, while the second current weight of the target dimensions corresponding to the other sub-loss values ​​is decreased, so as to obtain the target weight of each target dimension, wherein the other sub-loss values ​​are the loss values ​​included among the N first sub-loss values ​​other than the largest sub-loss value.

[0052] In an exemplary embodiment, the determining module 56 may determine the first loss value based on each first sub-loss value and each target weight in the following manner: determining the reciprocal of the product of each target weight and a first constant to obtain N first reciprocals; determining the first product of each first sub-loss value and the first reciprocal corresponding to the first sub-loss value to obtain the N first products; determining the Nth root of the product of the N target weights; determining the logarithm of the Nth root with a second constant as the base; and determining the first sum of the N first products and the logarithm as the first loss value.

[0053] In an exemplary embodiment, the determining module 56 may determine a second loss value based on the first feature map and the second feature map by: determining the mean square error between the first feature map and the second feature map; and determining the mean square error as the second loss value.

[0054] In an exemplary embodiment, the determining module 56 may determine a target loss value based on the first loss value and the second loss value by: determining a first weight corresponding to the first loss value and a second weight corresponding to the second loss value; determining a second product of the first loss value and the first weight; determining a third product of the second loss value and the second weight; and determining a second sum of the second product and the third product as the target loss value.

[0055] In one exemplary embodiment, the apparatus may be used to: train an initial model using a training dataset to obtain the initial student model before identifying the training data from the N target dimensions using an initial student model, wherein the training dataset includes the N sub-training datasets, the initial model includes the N initial second sub-models, and one of the sub-training datasets is used to train one of the initial second sub-models to obtain the second sub-model.

[0056] In one exemplary embodiment, the apparatus can also be used to identify a target image from the N target dimensions using the target network model after obtaining the target network model; and output the target recognition result of the target image in the N target dimensions.

[0057] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0058] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0059] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0060] Embodiments of the present invention also provide an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0061] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0062] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0063] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for training a model, characterized in that, include: The trained target teacher model is used to identify training data from N target dimensions, determining the first feature map of the training data and the first identification result of the training data in each target dimension. In the case of the target network model applied in the transportation field, the target dimensions include vehicle type, license plate, and vehicle orientation. The training data includes vehicle images. The process of using the trained target teacher model to identify training data from N target dimensions and determining the first feature map of the training data and the first identification result of the training data in each target dimension includes: using the target teacher model to identify vehicle images from the dimensions of license plate, vehicle type, and vehicle orientation; determining the first feature map of the vehicle image; and determining the license plate information, vehicle type information, and vehicle orientation information of the vehicle image. The target teacher model includes the N first sub-models, and one first sub-model is used to identify the training data from one of the target dimensions. The initial student model is used to identify the training data from the N target dimensions, and the second feature map of the training data and the second identification result of the training data in each target dimension are determined. The initial student model is a network model obtained after initial training. The initial student model includes the N second sub-models. One second sub-model is used to identify the training data from one target dimension. The target loss value of the initial student model is determined based on the first recognition result, the second recognition result, the first feature map, and the second feature map. If the target loss value does not meet the predetermined condition, the network parameters of the initial student model are updated until the target loss value meets the predetermined condition, thus obtaining the target network model. After obtaining the target network model, the method further includes: using the target network model to identify the target image from the N target dimensions; and outputting the target recognition result of the target image in the N target dimensions. Wherein, when the target network model is applied in the transportation field, the target image includes vehicle images; Determining the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map includes: determining a first loss value based on the first recognition result and the second recognition result; determining a second loss value based on the first feature map and the second feature map; and determining a target loss value based on the first loss value and the second loss value. The first feature map is the feature map of the last layer of the backbone network of the target teacher model, and the second feature map is the feature map of the last layer of the backbone network of the initial student model. Determining the second loss value based on the first feature map and the second feature map includes: compressing the dimension of the last layer feature map of the backbone network of the target teacher model into a first two-dimensional vector; compressing the dimension of the last layer feature map of the backbone network of the initial student model into a second two-dimensional vector; and determining the mean square error of the product of the first two-dimensional vector and the transpose of the first two-dimensional vector and the product of the second two-dimensional vector and the transpose of the second two-dimensional vector as the second loss value.

2. The method according to claim 1, characterized in that, Determining the first loss value based on the first identification result and the second identification result includes: Determine a first sub-identification result for each target dimension included in the first identification result and a second sub-identification result for each target dimension included in the second identification result; The first sub-loss value is determined based on the first sub-identification result and the second sub-identification result for each of the target dimensions; Determine the target weight corresponding to each of the target dimensions; The first loss value is determined based on each of the first sub-loss values ​​and each of the target weights.

3. The method according to claim 2, characterized in that, Determining the target weights for each of the target dimensions includes: In the case of determining the first loss value for the first time, the predetermined initial weights are determined as the target weights for each target dimension; Before determining the first loss value for the first time, the largest sub-loss value among the N first sub-loss values ​​is determined. The first current weight of the target dimension corresponding to the largest sub-loss value is increased according to a first predetermined method, and the second current weight of the target dimension corresponding to the other sub-loss values ​​is decreased to obtain the target weight of each target dimension. The other sub-loss values ​​are the loss values ​​other than the largest sub-loss value among the N first sub-loss values.

4. The method according to claim 3, characterized in that, Determining the first loss value based on each of the first sub-loss values ​​and each of the target weights includes: Determine the reciprocal of the product of each target weight and the first constant to obtain N first reciprocals; Determine the first product of each first sub-loss value and the first reciprocal corresponding to the first sub-loss value to obtain the N first products; Determine the Nth root of the product of the N target weights; Determine the logarithm of the Nth root with the second constant as the base; The first sum of the N first products and the logarithm is determined as the first loss value.

5. The method according to claim 1, characterized in that, Determining the target loss value based on the first loss value and the second loss value includes: Determine the first weight corresponding to the first loss value and the second weight corresponding to the second loss value; Determine the second product of the first loss value and the first weight; Determine the third product of the second loss value and the second weight; The second sum of the second product and the third product is determined as the target loss value.

6. The method according to claim 1, characterized in that, Before using the initial student model to identify the training data from the N target dimensions, the method further includes: An initial model is trained using a training dataset to obtain the initial student model. The training dataset includes the N sub-training datasets, and the initial model includes the N initial second sub-models. One of the sub-training datasets is used to train one of the initial second sub-models to obtain the second sub-model.

7. A training device for a model, characterized in that, include: The first recognition module is used to identify training data from N target dimensions using a trained target teacher model, and to determine the first feature map of the training data and the first recognition result of the training data in each target dimension. In the case of a target network model applied in the transportation field, the target dimensions include vehicle type, license plate, and vehicle orientation. The training data includes vehicle images. The first recognition module achieves this by using the trained target teacher model to identify training data from N target dimensions, determining the first feature map of the training data, and the first recognition result of the training data in each target dimension as follows: The target teacher model is used to identify vehicle images from the dimensions of license plate, vehicle type, and vehicle orientation; the first feature map of the vehicle image is determined; and the license plate information, vehicle type information, and vehicle orientation information of the vehicle image are determined. The target teacher model includes the N first sub-models, and one first sub-model is used to identify the training data from one of the target dimensions. The second recognition module is used to recognize the training data from the N target dimensions using the initial student model, determine the second feature map of the training data and the second recognition result of the training data in each target dimension, wherein the initial student model is a network model obtained after initial training, and the initial student model includes the N second sub-models, one of which is used to recognize the training data from one target dimension; The determining module is configured to determine the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map; The training module is used to update the network parameters of the initial student model when the target loss value does not meet the predetermined conditions, until the target loss value meets the predetermined conditions, thereby obtaining the target network model; The apparatus is further configured, after obtaining the target network model, to: identify the target image from the N target dimensions using the target network model; and output the target recognition result of the target image in the N target dimensions; Wherein, when the target network model is applied in the transportation field, the target image includes vehicle images; The determining module determines the target loss value of the initial student model based on the first recognition result, the second recognition result, the first feature map, and the second feature map in the following manner: determining a first loss value based on the first recognition result and the second recognition result; determining a second loss value based on the first feature map and the second feature map; and determining a target loss value based on the first loss value and the second loss value. The first feature map is the feature map of the last layer of the backbone network of the target teacher model, and the second feature map is the feature map of the last layer of the backbone network of the initial student model. The determining module determines the second loss value based on the first feature map and the second feature map in the following way: compressing the dimension of the last layer feature map of the backbone network of the target teacher model into a first two-dimensional vector; compressing the dimension of the last layer feature map of the backbone network of the initial student model into a second two-dimensional vector; and determining the mean square error of the product of the first two-dimensional vector and the transpose of the first two-dimensional vector and the product of the second two-dimensional vector and the transpose of the second two-dimensional vector as the second loss value.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 6.

Images