Knowledge Distillation Method and System for Object Recognition Network Based on Adaptive Weights

Abstract

The present invention relates to the field of target recognition technology, and provides a method and system for first knowledge distillation of a target recognition network based on adaptive weights. The method comprises: obtaining an input image; inputting the input image into a first target recognition network to obtain a target recognition result; the first target recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function comprises a first knowledge distillation weight, a first knowledge distillation sub-function; the product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value. The present invention can solve the problem that the training process of the first target recognition network as a student network is unstable and difficult to converge, eliminates the complex and inefficient hyperparameter setting process in traditional knowledge distillation, and obtains a first target recognition network with lower computing resource requirements and higher recognition efficiency to perform target recognition tasks.

Description

Technical Field

[0001] The present invention relates to the field of target recognition technology, and in particular to a target recognition network knowledge distillation method and system based on adaptive weights. Background Art

[0002] In many situations, images convey information that is richer, more tangible, and more specific than other forms of information. The collaboration between the human eye and brain enables us to acquire, process, and understand visual information, and humans use vision to perceive the external environment with high efficiency. Vision is the primary means by which humans acquire external information, and for computers to achieve intelligent intelligence, the ability to process image information is essential. In recent years, image data processing, characterized by large volumes of graphics, images, and videos, has been widely used in fields such as medicine, transportation, and industrial automation.

[0003] Existing technologies provide some target recognition methods based on neural networks. However, due to the inherent properties of neural networks, the accuracy of target recognition is highly correlated with the complexity of the network. As a result, traditional neural network methods cannot achieve good results in some scenarios with limited computing resources (or high recognition speed requirements).

[0004] To solve this problem, some methods based on knowledge distillation to compress neural network models have been proposed. Knowledge distillation (KD) is a model compression method and a training method based on the "teacher-student network idea" and is widely used in industry.

[0005] However, in existing knowledge distillation methods:

[0006] 1. It is difficult to match the teacher network (Teacher Model) and the student network (Student Model). Poor matching between the teacher network and the student network may lead to unstable training process and non-convergence problems.

[0007] 2. The KD algorithm is highly sensitive to hyperparameters, and the setting of hyperparameters relies on trial and error experience.

[0008] It can be seen that the existing knowledge distillation methods have problems such as harsh application conditions, insufficient algorithm stability, and high trial and error costs.

[0009] Therefore, how to compress neural networks based on a more stable and easy-to-converge knowledge distillation method to achieve better target recognition effects (lower computing resource requirements and higher recognition efficiency) has become a technical problem that needs to be solved urgently in the industry. Summary of the Invention

[0010] The present invention provides a target recognition network knowledge distillation method and system based on adaptive weights, which are used to solve the defects of the existing knowledge distillation method in the art, such as harsh application conditions, insufficient algorithm stability, and high trial and error costs, and achieve target recognition with lower computing resource requirements and higher efficiency.

[0011] The present invention provides a method for object recognition network knowledge distillation based on adaptive weights, comprising:

[0012] Get the input image;

[0013] Inputting the input image into a first object recognition network to obtain an object recognition result;

[0014] The first object recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight, and a first truth value sub-function;

[0015] The first truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value;

[0016] The first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label, and the teacher loss function;

[0017] The product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0018] According to the adaptive weight-based object recognition network knowledge distillation method provided by the present invention, the first object recognition network is obtained after the second joint training and the first joint training in sequence;

[0019] The second joint training is performed based on the second sample, the second label and the second loss function; the second loss function includes a second knowledge distillation weight, a second knowledge distillation sub-function, a second truth weight and a second truth sub-function;

[0020] The second truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the second sample as input and the second label; the second truth value weight is a third set value;

[0021] The second knowledge distillation sub-function can quantify the difference between the output of the first target recognition network as a student network with the second sample as input and the output of the second target recognition network as a teacher network with the second sample as input; the second knowledge distillation weight is a fourth set value.

[0022] According to a method for knowledge distillation of an object recognition network based on adaptive weights provided by the present invention, the first object recognition network includes a first branch capable of reasoning a first task and a second branch capable of reasoning a second task; the second object recognition network includes a third branch capable of reasoning the first task and a fourth branch capable of reasoning the second task;

[0023] The first truth sub-function includes a first task truth sub-function and a second task truth sub-function; the first task truth sub-function is capable of quantifying the difference between the output of the first branch and the first task truth value in the first label; the second task truth sub-function is capable of quantifying the difference between the output of the second branch and the second task truth value in the first label;

[0024] The first truth value weight includes the first task truth value weight and the second task truth weight; the first task truth value weight and the second task truth weight are the fifth set value and the sixth set value respectively;

[0025] The first knowledge distillation sub-function includes a first task knowledge distillation sub-function and a second task knowledge distillation sub-function; the first task knowledge distillation sub-function is a nonlinear function of the first branch output tensor and the third branch output tensor; the second task knowledge distillation sub-function is a nonlinear function of the second branch output tensor and the fourth branch output tensor;

[0026] The first knowledge distillation weight includes the first task knowledge distillation weight and the second task knowledge distillation weight; the product of the first task knowledge distillation weight and the scalar part of the first task knowledge distillation sub-function is the seventh set value; the product of the second task knowledge distillation weight and the scalar part of the second task knowledge distillation sub-function is the eighth set value.

[0027] According to the adaptive weight-based target recognition network knowledge distillation method provided by the present invention, the learning rates of the first joint training and the second joint training decrease as the number of training times increases.

[0028] According to a target recognition network knowledge distillation method based on adaptive weights provided by the present invention, the second set value is a constant hyperparameter determined based on the difference between the output of the second target recognition network with the first sample as input and the first label.

[0029] According to a method for knowledge distillation of an object recognition network based on adaptive weights provided by the present invention, the step of inputting the input image into a first object recognition network to obtain an object recognition result includes:

[0030] An input image including a traffic light is input into a first object recognition network to obtain an object recognition result including a traffic light position and / or a traffic light category.

[0031] The present invention also provides a target recognition network knowledge distillation system based on adaptive weights, comprising:

[0032] An acquisition module, used to acquire an input image;

[0033] a recognition module, configured to input the input image into a first target recognition network to obtain a target recognition result;

[0034] The first object recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight, and a first truth value sub-function;

[0035] The first truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value;

[0036] The first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label, and the teacher loss function;

[0037] The product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0038] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the steps of the target recognition network knowledge distillation method based on adaptive weights as described above are implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for target recognition network knowledge distillation based on adaptive weights.

[0040] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for target recognition network knowledge distillation based on adaptive weights.

[0041] The adaptive weight-based target recognition network knowledge distillation method and system provided by the present invention, by setting the first knowledge distillation weight as an adaptive weight related to the scalar part of the first knowledge distillation sub-function, can use the tensor part of the first knowledge distillation sub-function as the knowledge distillation loss and perform backpropagation to adjust the parameters of the first target recognition network, thereby performing knowledge distillation on the second target recognition network, that is, the teacher network, and solving the problem that the training process of the first target recognition network as the student network is unstable and difficult to converge, eliminating the complex and inefficient hyperparameter setting process in traditional knowledge distillation, and obtaining a first target recognition network with lower computing resource requirements and higher recognition efficiency after compression to perform target recognition tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 Schematic diagram of the process of the target recognition network knowledge distillation method based on adaptive weights provided by the present invention;

[0044] Figure 2 is a schematic diagram of the structure of a student network provided by an embodiment of the present invention;

[0045] Figure 3 Schematic diagram of the structure of the teacher network provided by an embodiment of the present invention;

[0046] Figure 4 Schematic diagram of the knowledge distillation process provided by an embodiment of the present invention;

[0047] Figure 5 Schematic diagram of the structure of the target recognition network knowledge distillation system based on adaptive weights provided by the present invention;

[0048] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention.

[0049] Reference numerals:

[0050] 1: Acquisition module; 2: Identification module; 610: Processor;

[0051] 620: Communication interface; 630: Memory; 640: Communication bus. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0053] The following combination Figure 1-Figure 3 The present invention describes the object recognition network knowledge distillation method based on adaptive weights.

[0054] like Figure 1 As shown, an embodiment of the present invention provides a method for object recognition network knowledge distillation based on adaptive weights, including:

[0055] Step 200, obtaining an input image;

[0056] Step 400: input the input image into a first object recognition network to obtain an object recognition result;

[0057] The first object recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight, and a first truth value sub-function;

[0058] The first truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value;

[0059] The first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label, and the teacher loss function;

[0060] The product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0061] In this embodiment, the target recognition result obtained in step 400 may be a target detection result and / or a target classification result; for example, for road image target recognition in an autonomous driving or assisted driving scenario, if the input image obtained in step 200 is a picture directly acquired by an image acquisition device and includes road information, traffic sign information, traffic light information, and obstacle information, then the output result may be a target classification result for a specific category of target (road information, traffic sign information, traffic light information, or obstacle information), or a target detection result not for a specific category of target, or a target detection and classification result not for a specific category of target.

[0062] In this embodiment, the first knowledge distillation sub-function is in the form of a tensor.

[0063] In a preferred embodiment, the first truth sub-function is in the form of a tensor; the first setting value is dynamically set and changes with the number of training times; and the second setting value is a fixed value.

[0064] The beneficial effects of this embodiment are:

[0065] By setting the first knowledge distillation weight as an adaptive weight related to the scalar part of the first knowledge distillation sub-function, it is possible to use the tensor part of the first knowledge distillation sub-function as the knowledge distillation loss and perform backpropagation to adjust the parameters of the first target recognition network, thereby performing knowledge distillation on the second target recognition network, i.e., the teacher network, and solving the problem that the training process of the first target recognition network as the student network is unstable and difficult to converge, eliminating the complex and inefficient hyperparameter setting process in traditional knowledge distillation, and obtaining a first target recognition network with lower computing resource requirements and higher recognition efficiency after compression to perform target recognition tasks.

[0066] According to the above embodiment, in this embodiment:

[0067] The first target recognition network is obtained after the second joint training and the first joint training in sequence;

[0068] The second joint training is performed based on the second sample, the second label and the second loss function; the second loss function includes a second knowledge distillation weight, a second knowledge distillation sub-function, a second truth weight and a second truth sub-function;

[0069] The second truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the second sample as input and the second label; the second truth value weight is a third set value;

[0070] The second knowledge distillation sub-function can quantify the difference between the output of the first target recognition network as a student network with the second sample as input and the output of the second target recognition network as a teacher network with the second sample as input; the second knowledge distillation weight is a fourth set value.

[0071] In this embodiment, through the second joint training, the fourth set value is used as the second knowledge distillation weight, so that the first object recognition network can perform knowledge distillation according to the degree corresponding to the fourth set value. Although this process may have problems such as unstable training and difficulty in parameter convergence, the first object recognition network can still learn the knowledge of the second object recognition network to a large extent.

[0072] Subsequently, through the first joint training, the adaptive first knowledge distillation weights are used to control the influence of the second target recognition network output on the overall loss function, making the training process more stable and the parameters easier to converge.

[0073] Furthermore, the learning rates of the first joint training and the second joint training decrease as the number of training times increases.

[0074] The second setting value is a constant hyperparameter determined based on a difference between an output of the second target recognition network with the first sample as input and the first label.

[0075] In this embodiment, the dynamic setting of the learning rate is also conducive to improving the convergence speed of model training.

[0076] The second set value is set with reference to the difference between the output of the second target recognition network with the first sample as input and the first label, which can further improve the training stability.

[0077] According to any of the above embodiments, in this embodiment:

[0078] The step of inputting the input image into a first object recognition network to obtain an object recognition result comprises:

[0079] An input image including a traffic light is input into a first object recognition network to obtain an object recognition result including a traffic light position and / or a traffic light category.

[0080] That is, this embodiment provides a target recognition method for identifying traffic light categories. In traffic light category recognition scenarios and other target recognition fields, due to the needs of task settings, the target recognition network may have multiple task branches. For similar multi-branch networks, the adaptive weight knowledge distillation method of the present invention can also be applied. The following will be explained using a target recognition network including two branches as an example. A target recognition network with more branches can be obtained by analogy through the following description.

[0081] The first target recognition network includes a first branch capable of reasoning about a first task and a second branch capable of reasoning about a second task; the second target recognition network includes a third branch capable of reasoning about the first task and a fourth branch capable of reasoning about the second task;

[0082] The first truth sub-function includes a first task truth sub-function and a second task truth sub-function; the first task truth sub-function is capable of quantifying the difference between the output of the first branch and the first task truth value in the first label; the second task truth sub-function is capable of quantifying the difference between the output of the second branch and the second task truth value in the first label;

[0083] The first truth value weight includes the first task truth value weight and the second task truth weight; the first task truth value weight and the second task truth weight are the fifth set value and the sixth set value respectively;

[0084] The first knowledge distillation sub-function includes a first task knowledge distillation sub-function and a second task knowledge distillation sub-function; the first task knowledge distillation sub-function is a nonlinear function of the first branch output tensor and the third branch output tensor; the second task knowledge distillation sub-function is a nonlinear function of the second branch output tensor and the fourth branch output tensor;

[0085] The first knowledge distillation weight includes the first task knowledge distillation weight and the second task knowledge distillation weight; the product of the first task knowledge distillation weight and the scalar part of the first task knowledge distillation sub-function is the seventh set value; the product of the second task knowledge distillation weight and the scalar part of the second task knowledge distillation sub-function is the eighth set value.

[0086] In a preferred embodiment, Figure 2 As shown in , the first target recognition network as the student network includes an input layer, a feature extraction layer based on mobilenetv2, and a decoder layer including a first branch and a second branch; Figure 3 As shown in , the second target network as the teacher network includes an input layer, a feature extraction layer based on resnet50, and a decoder layer including a third branch and a fourth branch; Figure 4As shown in the figure, the first joint training is based on a small batch of image groups, i.e., batches. After the batch images are input into the teacher network and the student network respectively, the knowledge distillation loss (i.e., knowledge distillation loss function) of the adaptive weights is calculated according to the output tensors of the two, and the parameters are updated in the student network based on the loss to achieve the purpose of knowledge distillation.

[0087] In this embodiment, the parameter update is determined by the loss of the current batch data and the learning rate (the learning rate multiplied by the back-propagation gradient of the loss). Since the product of the scalar part of the distillation loss and the weight has been set to a fixed value, the parameter update of the student network will be more stable. For example, if the learning rate remains unchanged, the pace of each parameter update will be consistent. Therefore, this embodiment and the above-mentioned embodiment methods can obtain a more stable training process, avoiding the problem of model non-convergence caused by improper hyperparameter adjustment.

[0088] The target recognition network knowledge distillation device based on adaptive weights provided by the present invention is described below. The target recognition network knowledge distillation device based on adaptive weights described below and the target recognition network knowledge distillation method based on adaptive weights described above can be referenced to each other.

[0089] like Figure 5 As shown, an embodiment of the present invention further provides a target recognition network knowledge distillation system based on adaptive weights, comprising:

[0090] Acquisition module 1, used to acquire input images;

[0091] Recognition module 2, used for inputting the input image into a first target recognition network to obtain a target recognition result;

[0092] The first object recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight, and a first truth value sub-function;

[0093] The first truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value;

[0094] The first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label, and the teacher loss function;

[0095] The product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0096] Furthermore, the first target recognition network is obtained after the second joint training and the first joint training in sequence;

[0097] The second joint training is performed based on the second sample, the second label and the second loss function; the second loss function includes a second knowledge distillation weight, a second knowledge distillation sub-function, a second truth weight and a second truth sub-function;

[0098] The second truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the second sample as input and the second label; the second truth value weight is a third set value;

[0099] The second knowledge distillation sub-function can quantify the difference between the output of the first target recognition network as a student network with the second sample as input and the output of the second target recognition network as a teacher network with the second sample as input; the second knowledge distillation weight is a fourth set value.

[0100] The first target recognition network includes a first branch capable of reasoning about a first task and a second branch capable of reasoning about a second task; the second target recognition network includes a third branch capable of reasoning about the first task and a fourth branch capable of reasoning about the second task;

[0101] The first truth sub-function includes a first task truth sub-function and a second task truth sub-function; the first task truth sub-function is capable of quantifying the difference between the output of the first branch and the first task truth value in the first label; the second task truth sub-function is capable of quantifying the difference between the output of the second branch and the second task truth value in the first label;

[0102] The first truth value weight includes the first task truth value weight and the second task truth weight; the first task truth value weight and the second task truth weight are the fifth set value and the sixth set value respectively;

[0103] The first knowledge distillation sub-function includes a first task knowledge distillation sub-function and a second task knowledge distillation sub-function; the first task knowledge distillation sub-function is a nonlinear function of the first branch output tensor and the third branch output tensor; the second task knowledge distillation sub-function is a nonlinear function of the second branch output tensor and the fourth branch output tensor;

[0104] The first knowledge distillation weight includes the first task knowledge distillation weight and the second task knowledge distillation weight; the product of the first task knowledge distillation weight and the scalar part of the first task knowledge distillation sub-function is the seventh set value; the product of the second task knowledge distillation weight and the scalar part of the second task knowledge distillation sub-function is the eighth set value.

[0105] The learning rates of the first joint training and the second joint training decrease as the number of training times increases.

[0106] The second setting value is a constant hyperparameter determined based on a difference between an output of the second target recognition network with the first sample as input and the first label.

[0107] The identification module 2 includes:

[0108] The traffic light recognition unit is configured to input an input image including a traffic light into a first object recognition network to obtain an object recognition result including a traffic light position and / or a traffic light category.

[0109] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6As shown, the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute the adaptive weight-based target recognition network knowledge distillation method, which includes: obtaining an input image; inputting the input image into a first target recognition network to obtain a target recognition result; the first target recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth weight and a first truth sub-function; the first truth sub-function can quantify the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value; the first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label and the teacher loss function; the product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0110] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0111] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the adaptive weight-based target recognition network knowledge distillation method provided by the above methods, the method including: obtaining an input image; inputting the input image into a first target recognition network to obtain a target recognition result; the first target recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight and a first truth value sub-function. function; the first truth sub-function can quantify the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value; the first knowledge distillation sub-function is a nonlinear function of the first tensor and the second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, teacher label and teacher loss function; the product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0112] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the adaptive weight-based target recognition network knowledge distillation method provided by the above methods, the method comprising: obtaining an input image; inputting the input image into a first target recognition network to obtain a target recognition result; the first target recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight and a first truth value sub-function; the first truth sub-function can Quantify the difference between the output of the first target recognition network with the first sample as input and the first label; the first true value weight is a first set value; the first knowledge distillation sub-function is a nonlinear function of the first tensor and the second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, teacher label and teacher loss function; the product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A knowledge distillation method for target recognition network based on adaptive weights, characterized in that: include: Get the input image; Inputting the input image into a first object recognition network to obtain an object recognition result; The first object recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight, and a first truth value sub-function; The first truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value; The first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label, and the teacher loss function; The product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value; The first target recognition network includes a first branch capable of reasoning about a first task and a second branch capable of reasoning about a second task; the second target recognition network includes a third branch capable of reasoning about the first task and a fourth branch capable of reasoning about the second task; The first truth sub-function includes a first task truth sub-function and a second task truth sub-function; The first task truth sub-function is capable of quantifying the difference between the output of the first branch and the first task truth value in the first label; The second task truth sub-function is capable of quantifying the difference between the output of the second branch and the second task truth value in the first label; The first truth value weight includes the first task truth value weight and the second task truth weight; the first task truth value weight and the second task truth weight are the fifth set value and the sixth set value respectively; The first knowledge distillation sub-function includes a first task knowledge distillation sub-function and a second task knowledge distillation sub-function; the first task knowledge distillation sub-function is a nonlinear function of the first branch output tensor and the third branch output tensor; the second task knowledge distillation sub-function is a nonlinear function of the second branch output tensor and the fourth branch output tensor; The first knowledge distillation weight includes the first task knowledge distillation weight and the second task knowledge distillation weight; the product of the first task knowledge distillation weight and the scalar part of the first task knowledge distillation sub-function is the seventh set value; the product of the second task knowledge distillation weight and the scalar part of the second task knowledge distillation sub-function is the eighth set value.

2. The method for target recognition network knowledge distillation based on adaptive weights according to claim 1, characterized in that: The first target recognition network is obtained after the second joint training and the first joint training in sequence; The second joint training is performed based on the second sample, the second label and the second loss function; the second loss function includes a second knowledge distillation weight, a second knowledge distillation sub-function, a second truth weight and a second truth sub-function; The second truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the second sample as input and the second label; the second truth value weight is a third set value; The second knowledge distillation sub-function can quantify the difference between the output of the first target recognition network as a student network with the second sample as input and the output of the second target recognition network as a teacher network with the second sample as input; the second knowledge distillation weight is a fourth set value.

3. The method for target recognition network knowledge distillation based on adaptive weights according to claim 2, characterized in that: The learning rates of the first joint training and the second joint training decrease as the number of training times increases.

4. The method for target recognition network knowledge distillation based on adaptive weights according to claim 1, characterized in that: The second setting value is a constant hyperparameter determined based on a difference between an output of the second target recognition network with the first sample as input and the first label.

5. The method for target recognition network knowledge distillation based on adaptive weights according to any one of claims 1 to 4, characterized in that: The step of inputting the input image into a first object recognition network to obtain an object recognition result comprises: An input image including a traffic light is input into a first object recognition network to obtain an object recognition result including a traffic light position and / or a traffic light category.

6. A target recognition network knowledge distillation system based on adaptive weights, characterized by: include: An acquisition module, used to acquire an input image; a recognition module, configured to input the input image into a first target recognition network to obtain a target recognition result; The first object recognition network is obtained after a first joint training; the first joint training is performed based on a first sample, a first label, and a first loss function; the first loss function includes a first knowledge distillation weight, a first knowledge distillation sub-function, a first truth value weight, and a first truth value sub-function; The first truth sub-function is capable of quantifying the difference between the output of the first target recognition network with the first sample as input and the first label; the first truth weight is a first set value; The first knowledge distillation sub-function is a nonlinear function of a first tensor and a second tensor; the first tensor is the output tensor of the first target recognition network as a student network with the first sample as input; the second tensor is the output tensor of the second target recognition network as a teacher network with the first sample as input; the second target recognition network is trained based on the teacher sample, the teacher label, and the teacher loss function; The product of the first knowledge distillation weight and the scalar part of the first knowledge distillation sub-function is a second set value; The first target recognition network includes a first branch capable of reasoning about a first task and a second branch capable of reasoning about a second task; the second target recognition network includes a third branch capable of reasoning about the first task and a fourth branch capable of reasoning about the second task; The first truth sub-function includes a first task truth sub-function and a second task truth sub-function; The first task truth sub-function is capable of quantifying the difference between the output of the first branch and the first task truth value in the first label; The second task truth sub-function is capable of quantifying the difference between the output of the second branch and the second task truth value in the first label; The first truth value weight includes the first task truth value weight and the second task truth weight; the first task truth value weight and the second task truth weight are the fifth set value and the sixth set value respectively; The first knowledge distillation sub-function includes a first task knowledge distillation sub-function and a second task knowledge distillation sub-function; the first task knowledge distillation sub-function is a nonlinear function of the first branch output tensor and the third branch output tensor; the second task knowledge distillation sub-function is a nonlinear function of the second branch output tensor and the fourth branch output tensor; The first knowledge distillation weight includes the first task knowledge distillation weight and the second task knowledge distillation weight; the product of the first task knowledge distillation weight and the scalar part of the first task knowledge distillation sub-function is the seventh set value; the product of the second task knowledge distillation weight and the scalar part of the second task knowledge distillation sub-function is the eighth set value.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the target recognition network knowledge distillation method based on adaptive weights as described in any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the target recognition network knowledge distillation method based on adaptive weights as described in any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the target recognition network knowledge distillation method based on adaptive weights as described in any one of claims 1 to 5 are implemented.

Images