Knowledge distillation method and device of detection model, robot and program product

Abstract

The invention discloses a knowledge distillation method and device for a detection model, a robot and a computer program product. The teacher model has a larger framework and more parameters, so that the performance and the feature extraction capability are stronger. In order to give full play to the advantages, the knowledge distillation method maximizes the feature extraction ability by expanding the scale of the second sample to train the teacher model based on the first sample. And then, inputting the first sample into a teacher model, inputting the second sample into a student model with a smaller framework and fewer parameters, and enabling the student model to effectively learn the knowledge of the teacher model through a knowledge distillation technology. Specifically, a total loss function is calculated by utilizing knowledge representation of a teacher model for a first sample and learning representation of a student model for a second sample, and the student model is iteratively optimized to gradually approach complex knowledge representation of the teacher model. Finally, a student model which has performance close to that of a teacher model and is lighter and less in resource occupation is obtained, a new thought is provided for deployment of the model in a resource-constrained environment, and the applicability of the model in scenes of mobile equipment, embedded systems and the like is improved.

Description

Technical Field

[0001] The present application belongs to the field of model compression technology, and in particular, relates to a knowledge distillation method for a detection model, a knowledge distillation device for a detection model, a robot, and a computer program product. Background Art

[0002] Knowledge distillation is an efficient model compression technology that transfers the knowledge extracted from the teacher model with a larger framework and more parameters to the student model with a smaller framework and fewer parameters in the form of soft labels and hard labels, so that it can accurately simulate the behavior of the teacher model. In this process, the student model not only retains the high performance of the teacher model, but also significantly reduces the computational complexity and memory usage. The trained student model provides a new idea for model deployment in resource-constrained environments due to its lightweight and high efficiency, and can be widely used in mobile devices, embedded systems, etc.

[0003] However, traditional knowledge distillation methods usually assume that the feature distribution of the teacher model and the student model is consistent, ignoring the difference in feature extraction capabilities between the two. This assumption results in the failure to maximize the use of the powerful feature extraction capabilities of the teacher model, and hinders the student model from fully absorbing the complex knowledge of the teacher model. Summary of the invention

[0004] The present application provides a knowledge distillation method for a detection model, a knowledge distillation device for a detection model, a robot and a computer program product, which can fully utilize the feature extraction capability of a teacher model to improve the learning effect and semantic expression capability of a student model on the feature representation of the teacher model, thereby narrowing the performance gap between the student model and the teacher model, so that the student model can achieve lightweight while maintaining high performance.

[0005] In a first aspect, the present application provides a knowledge distillation method for a detection model, wherein the detection model includes a student model; the knowledge distillation method includes:

[0006] The following steps are repeated until a converged student model is obtained:

[0007] The first sample and the second sample are respectively input into the teacher model and the student model for training, so as to obtain the knowledge representation of the teacher model and the learning representation of the student model respectively; the first sample is obtained by scaling the second sample;

[0008] Calculate the total loss based on the knowledge representation and the learned representation;

[0009] The iterative student model is optimized based on the total loss.

[0010] In some embodiments, the knowledge representation includes a first intermediate layer feature of the teacher model and a first probability distribution of the target in the first sample; the learned representation includes a second intermediate layer feature of the student model and a second probability distribution of the target in the first sample; the total loss includes a distillation loss;

[0011] Accordingly, the total loss is calculated based on the knowledge representation and the learned representation, including:

[0012] calculating a first distillation loss based on the first intermediate layer characteristic and the second intermediate layer characteristic;

[0013] A second distillation loss is calculated based on the first probability distribution and the second probability distribution.

[0014] In some embodiments, both the teacher model and the student model perform at least two stages of feature extraction; the student model is provided with a scaling module after each stage; the scaling module is used to align the scale of the feature map of the current stage of the student model with the scale of the feature map of the corresponding stage of the teacher model;

[0015] The first intermediate layer features include the first feature map obtained by the teacher model after feature extraction at each stage;

[0016] The second intermediate layer features include the second feature map obtained by the student model after feature extraction and scale scaling at each stage;

[0017] Accordingly, calculating the first distillation loss based on the first intermediate layer characteristic and the second intermediate layer characteristic includes:

[0018] At each stage, the corresponding stage loss is calculated based on the corresponding first feature map and the corresponding second feature map;

[0019] The first distillation loss was calculated based on the stage losses of each stage.

[0020] In some embodiments, before the first sample and the second sample are input into the teacher model and the student model respectively, the method further includes:

[0021] Preprocessing the first sample and the second sample by specifying an enhancement operation to introduce self-supervised learning to the teacher model and the student model respectively based on the specified enhancement operation; both the teacher model and the student model perform aggregation task detection;

[0022] The total loss also includes the aggregated task loss; accordingly, the total loss is calculated based on the knowledge representation and the learned representation, including:

[0023] The distillation loss and aggregation task loss are calculated based on the knowledge representation and the learned representation.

[0024] In some embodiments, preprocessing the first sample and the second sample based on a specified enhancement operation includes:

[0025] Performing a set of specified enhancement operations on the first sample and the second sample respectively to obtain a first enhanced sample corresponding to the first sample and a second enhanced sample corresponding to the second sample;

[0026] Accordingly, the knowledge representation further includes the first prediction label of the aggregation task, and the learning representation further includes the second prediction label of the aggregation task. The first sample and the second sample are respectively input into the teacher model and the student model for training, and the knowledge representation of the teacher model and the learning representation of the student model are respectively obtained, including:

[0027] The first enhanced sample and the second enhanced sample are respectively input into the teacher model and the student model for training, and a first predicted label and a second predicted label are obtained;

[0028] The aggregate task loss is determined based on the following steps:

[0029] The aggregation task loss is calculated based on the true label of the aggregation task, the first predicted label, and the second predicted label.

[0030] In some embodiments, the specified enhancement operation includes at least one of a rotation operation, a blurring process, and a color disturbance, and a group of specified enhancement operations includes specified enhancement operations of different degrees.

[0031] In a second aspect, the present application provides a target detection method, which is applied to a robot, wherein the robot is deployed with a trained student detection model obtained by the knowledge distillation method of the detection model described in any embodiment of the first aspect; the target detection method comprises:

[0032] In the process of performing the task, the detection image is obtained;

[0033] Perform target detection on the detection image through the student detection model to obtain the target detection result;

[0034] Based on the target detection results and the current task status of the robot, the robot is controlled to operate.

[0035] In a third aspect, the present application provides a knowledge distillation device for a detection model, characterized in that the detection model includes a student model; the knowledge distillation device:

[0036] The following modules are triggered cyclically until a converged student model is obtained:

[0037] A training module, used for inputting the first sample and the second sample into the teacher model and the student model for training respectively, so as to obtain the knowledge representation of the teacher model and the learning representation of the student model respectively; the first sample is obtained by scaling the second sample;

[0038] A computation module for computing distillation loss based on knowledge representation and learned representation;

[0039] An optimization module for iteratively optimizing the student model based on the distillation loss.

[0040] In a fourth aspect, the present application provides a robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of any embodiment of the first aspect when executing the computer program.

[0041] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method of any embodiment of the first aspect are implemented.

[0042] In a sixth aspect, the present application provides a computer program product, which includes a computer program. When the computer program is executed by one or more processors, it implements the steps of the method of any embodiment of the first aspect.

[0043] Compared with the prior art, the present application has the following beneficial effects: the teacher model has a larger architecture and more parameters, so it has stronger performance and feature extraction capabilities. In order to give full play to its advantages, the knowledge distillation method constructs a first sample training teacher model by expanding the scale of the second sample to maximize its feature extraction capabilities. The first sample is input into the teacher model, and the second sample is input into the student model with a smaller architecture and fewer parameters. Through the knowledge distillation technology, the student model effectively learns the knowledge of the teacher model.

[0044] Specifically, by comparing the teacher model's knowledge representation of the first sample with the student model's learning representation of the second sample, the total loss function is calculated and the student model is iteratively optimized to gradually approach the complex knowledge representation of the teacher model. Ultimately, through this knowledge distillation method, a student model with performance close to that of the teacher model but with a lighter weight and lower resource usage can be obtained, providing an efficient solution for model deployment in resource-constrained environments, especially in scenarios such as mobile devices and embedded systems.

[0045] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 It is a schematic diagram of the flow of the knowledge distillation method of the detection model provided in the embodiment of the present application;

[0048] Figure 2 is a schematic diagram of the process of cross-scale knowledge distillation provided in an embodiment of the present application;

[0049] Figure 3 is a comparative schematic diagram of two training methods provided in the embodiment of the present application;

[0050] Figure 4 It is a schematic diagram of the structure of the knowledge distillation framework of the detection model provided in the embodiment of the present application;

[0051] Figure 5 It is a flowchart of the target detection method provided in the embodiment of the present application;

[0052] Figure 6 It is a schematic diagram of the structure of the knowledge distillation device of the detection model provided in the embodiment of the present application;

[0053] Figure 7 It is a schematic diagram of the structure of the robot provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] Knowledge Distillation (KD) is a model compression technology that aims to transfer the knowledge in a superior but large and complex teacher model to a smaller and lighter student model in some form, so that the student model can be as close to the teacher model as possible in performance, while significantly reducing computational complexity and resource usage.

[0056] In related technologies, traditional knowledge distillation methods usually assume that the feature distribution of the teacher model and the student model is consistent, ignoring the difference in feature extraction capabilities between the two. This assumption results in the failure to maximize the use of the powerful feature extraction capabilities of the teacher model, and hinders the student model from fully absorbing the complex knowledge of the teacher model.

[0057] In order to solve this problem, this application proposes a knowledge distillation method for a detection model, which can make full use of the feature extraction capability of the teacher model to improve the learning effect and semantic expression capability of the student model on the teacher model feature representation, thereby narrowing the performance gap between the student model and the teacher model, so that the student model can achieve lightweight while maintaining high performance. The control method proposed in this application will be described below through specific embodiments.

[0058] The knowledge distillation method of the detection model provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, vehicle-mounted equipment, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0059] In order to illustrate the technical solution proposed in this application, each embodiment will be described below with an electronic device as the execution subject.

[0060] Figure 1 A schematic flow chart of the knowledge distillation method for the detection model provided in the present application is shown. The performance of the generally deployed student models is stable and can achieve the expected results, so the purpose of the knowledge distillation method for the detection model is to obtain a converged student model. Among them, the converged student model refers to a model in which, through continuous iterative optimization during the knowledge distillation training process, its parameters gradually stabilize, the loss function tends to be stable, and ultimately the output results of the model reach the expected performance target, and no longer show obvious fluctuations or error decreases. At this point, it is considered that the student model has successfully learned the knowledge transferred by the teacher model, such as fine-grained features and class probability distributions, and has shown accuracy and performance close to the target on the validation set or test set, indicating that the model training has been completed and has good generalization ability and practical application effects. Therefore, the electronic device can cyclically perform the following steps to achieve this purpose:

[0061] Step 110: The electronic device inputs the first sample and the second sample into the teacher model and the student model for training, respectively, to obtain the knowledge representation of the teacher model and the learning representation of the student model.

[0062] The network structure of the student model is similar to that of the teacher model, but the difference is that the student model is more lightweight, has a smaller parameter scale, less network depth and width, and significantly reduces computational complexity, making it more suitable for resource-constrained scenarios such as mobile devices or embedded systems. In contrast, the teacher model has a larger parameter scale and a more complex network structure, with stronger computing power and the ability to extract richer features. It is usually used as a guide and only plays a role in the training phase. The student model learns the knowledge of the teacher model through knowledge distillation, and its performance is as close to the teacher model as possible, while achieving high efficiency and lightweight to meet the needs of actual applications.

[0063] In the knowledge distillation process, the teacher model can be regarded as an instructor, which mainly plays a role in the training stage, while the student model after distillation training is used as the final deployed model and is applied to resource-constrained scenarios, such as mobile devices such as mobile robots, embedded systems or edge computing environments.

[0064] In order to give full play to the advantages of the teacher model, the scale of the samples input to the teacher model and the student model is adjusted, that is, the scale of the samples input to the teacher model is larger than that of the samples input to the student model. This adjustment allows the teacher model to extract more fine-grained features without changing the original semantic features of the same sample image, maximizing its feature extraction ability. Accordingly, the student model can more effectively obtain detailed knowledge by learning the rich feature representation output by the teacher model, thereby improving its own performance.

[0065] For example, an image interpolation technique, such as nearest neighbor interpolation, bilinear interpolation, or cubic interpolation, may be used to double the width and height of the first sample to obtain a second sample with a larger scale.

[0066] Taking the training of a model used for classifying and identifying delivery items on a delivery robot's tray as an example, the first sample may be a picture of a dessert on a plate.

[0067] For example, for a first sample with a resolution of 256×256, it can be scaled up, for example, to a second sample with an image resolution of 512×512, where the first sample is used to input the student model and the second sample is used to input the teacher model. Based on this, the teacher model can capture more detailed edges, textures, and local features from large-scale images. The student model learns on the original image to obtain feature representation, which can provide a data basis for efficient knowledge distillation and performance improvement.

[0068] Accordingly, after the first sample image and the second sample image are respectively input into the teacher model and the student model for training, the teacher model can generate knowledge representations for imparting to the student model, such as fine-grained features and probability distributions of targets (soft labels), which contain rich semantic information extracted by the teacher model based on the extended deep understanding of the first sample. The student model generates learning representations through the current model parameters, which represents the student model's understanding of the first sample and the extracted semantic information based on the current learning.

[0069] Step 120: The electronic device calculates the total loss based on the knowledge representation and the learning representation.

[0070] Step 130: The electronic device optimizes the iterative student model based on the total loss.

[0071] In order to enable the student model to gradually master the feature extraction ability and behavioral performance of the teacher model, the representations of the two are subsequently compared and optimized through knowledge distillation, so that the student model can efficiently learn and approximate the knowledge expression of the teacher model.

[0072] Specifically, the training of the student model is guided by calculating the total loss function between the knowledge representation of the teacher model (such as fine-grained features and soft labels) and the learned representation of the student model (such as current features and prediction results).

[0073] Exemplarily, the total loss may include distillation loss and supervision loss; distillation loss is used to measure the difference between the output of the teacher model and the student model, such as using KL divergence to constrain the probability distribution of soft labels, or aligning feature maps through mean square error; supervision loss is based on the true label, constraining the error between the predicted label of the student model and the true label.

[0074] For example, assume that there are four types of dishes that the delivery robot needs to recognize, including birthday cakes, sandwiches, hamburgers, and egg tarts. Then the dish category corresponding to the dish included in the training sample is the real data label of the image. For the training sample containing egg tarts, its real label is egg tarts. If the student model predicts the label of the training sample as hamburger, there is an error between the predicted label and the real label. This error can be measured and minimized through the supervision loss, thereby optimizing the performance of the student model.

[0075] The electronic device iteratively optimizes the parameters of the student model through total loss, gradually narrowing the gap between it and the teacher model in feature extraction and behavioral performance, and ultimately obtaining a lightweight student model with performance close to that of the teacher model and lower computational complexity, providing an efficient and feasible solution for actual deployment.

[0076] In this embodiment, the electronic device allows the teacher model and the student model to process the second sample and the first sample with the same semantic features but across scales, which can give full play to the feature extraction advantages of the teacher model and help the student model learn fine-grained and rich knowledge, thereby improving its performance. Specifically, the teacher model extracts finer-grained features and richer semantic information through the large-scale second sample to generate a knowledge representation; the student model extracts the corresponding features and semantic information from the small-scale first sample based on the current model parameters to generate a learning representation. Subsequently, by calculating the loss function between the knowledge representation and the learning representation, the student model is iteratively optimized to gradually approach the feature extraction capability and behavioral performance of the teacher model, thereby achieving performance improvement and model lightweighting, and adapting to actual deployment scenarios with limited resources.

[0077] For example, for mobile robots, operations such as navigation and obstacle avoidance are required, and their computing resources are limited. When the robot wants to expand other functions, the computing resources that can be used are very limited. For example, by setting a camera above the load-bearing device (tray, storage bin), the load-bearing state is detected according to the image taken by the camera, and the robot adapts to the detection results based on the robot task state. For example, self-pickup recognition (whether the items at the delivery target point have been taken away, if taken away, the next delivery task or return is automatically executed), receipt retention reminder (whether the receipt at the delivery target point is stranded, if so, a reminder is given to take it away), return plate detection (whether there is a dirty plate on the load-bearing device that needs to be cleaned in the cleaning area, if so, the robot navigates to the cleaning area), etc. These solutions can greatly improve the utilization rate and work efficiency of the robot, and can also improve the customer experience of users, but they require computing resources. The student model trained in this application has the characteristics of high performance and lightweight, and is particularly suitable for image recognition and classification detection of mobile robots.

[0078] In some embodiments, the teacher model can be pre-trained based on the second sample, or can be trained synchronously with the student model. A pre-trained teacher model is preferred, which can ensure that the teacher model has strong robustness, reduce the transfer of erroneous knowledge, and output high-quality knowledge, effectively improve the distillation efficiency, and improve the generalization ability of the student model.

[0079] In some embodiments, during the knowledge distillation process, if the soft labels of the teacher model are too abstract, it will be difficult to fully reflect the knowledge in the reasoning process, which will make it difficult for the student model to effectively learn the reasoning mode of the teacher model. To ensure that the student model can fully learn the reasoning mode of the teacher model, the knowledge representation may include the intermediate layer features extracted by the teacher model and the probability distribution of the target in the first sample; accordingly, the learning representation includes the intermediate layer features extracted by the student model and the probability distribution of the target in the second sample. For ease of description, the intermediate layer features and probability distribution of the teacher model are respectively recorded as the first intermediate layer features and the first probability distribution, and the intermediate layer features and probability distribution of the student model are respectively recorded as the second intermediate layer features and the second probability distribution. In this way, by comparing and optimizing the two representations, it can help the student model learn the reasoning process of the teacher model more effectively.

[0080] Specifically, the total loss may include a distillation loss, and the distillation loss may include a first distillation loss and a second distillation loss. Thus, the electronic device may calculate the distillation loss by the following steps:

[0081] Step A1: The electronic device calculates a first distillation loss based on the first intermediate layer characteristic and the second intermediate layer characteristic.

[0082] Step A2: The electronic device calculates a second distillation loss based on the first probability distribution and the second probability distribution.

[0083] In order to make the performance of the student model close to that of the teacher model, in addition to the consistency in the final prediction probability, the alignment of the intermediate reasoning process is also crucial. In the above two steps, the electronic device first calculates the first distillation loss through the intermediate features of the two models to reflect the difference in feature learning between the student model and the teacher model; then, the second distillation loss is calculated through the probability distribution of the two models to measure the difference in target prediction ability between the student model and the teacher model. Finally, the electronic device iteratively optimizes the student model through the first distillation loss and the second distillation loss, allowing the student model to gradually learn and approach the reasoning process and prediction results of the teacher model, thereby improving its overall performance.

[0084] In some embodiments, in order to fully extract features from the image, both the teacher model and the student model perform at least two stages of feature extraction. Through staged feature extraction, the model can gradually capture different levels of information in the data, from low-level detail features to high-level abstract semantic features, enhancing the expressiveness and robustness of the model. This hierarchical learning not only helps the model to balance the capture of local and global features and improve the accuracy of the model, but also enhances its adaptability in the face of noise, deformation or missing information. Accordingly, in order to enable the student model to more fully learn the reasoning model of the teacher model, based on the characteristics of the staged learning of the two models, a staged knowledge distillation method is adopted to achieve model compression.

[0085] As can be seen from the above content, the training sample scales of the input teacher model and the student model are different. However, when performing staged knowledge distillation, the inconsistent scales of the feature maps extracted by the teacher model and the student model in each stage will bring challenges to the staged distillation, for example, it may reduce the learning effect of the student model. In order to smoothly and effectively perform staged knowledge distillation while maximizing the feature extraction ability of the teacher model, a rescale block can be set after each stage of the student model. The function of the rescale block is to align the scale of the feature map of the current stage of the student model with the scale of the feature map of the corresponding stage of the teacher model, so as to ensure that when distilling, the scale difference between the feature maps will not affect the learning process of the student model, and ensure that the knowledge transfer in the distillation process is more accurate and effective.

[0086] It can be considered that the staged knowledge distillation is to calculate the corresponding first distillation loss by aligning the intermediate features of each stage of the two models. Therefore, the first intermediate layer features include the first feature map obtained after the teacher model extracts the features at each stage; the second intermediate layer features include the second feature map obtained after the student model extracts the features at each stage and scales. Specifically, the electronic device can calculate the first distillation loss through the following steps:

[0087] Step B1: At each stage of the electronic device, a corresponding stage loss is calculated based on the corresponding first characteristic graph and the corresponding second characteristic graph.

[0088] Step B2: The electronic device calculates the first distillation loss based on the stage losses of each stage.

[0089] At each stage, the electronic device calculates the stage loss of that stage based on the corresponding first feature map (from the teacher model) and the second feature map (from the student model) to reflect the difference in feature extraction capabilities between the teacher and the student at the current stage. Subsequently, the electronic device aggregates the stage losses of all stages and calculates the first distillation loss. The first distillation loss calculated by this method can help the student model gradually learn and approach the feature extraction method of the teacher model at each stage, so that the student model can not only capture the detailed features of the data more accurately, enhance its understanding of complex patterns, but also ultimately improve the overall performance and robustness of the student model.

[0090] For example, see Figure 2 , Figure 2 The figure shows a schematic diagram of the process of cross-scale knowledge distillation. In this process, the teacher model can extract features from the large-scale second sample to supplement more detailed information, while the student model can supplement detailed information by learning the output of the teacher model on the basis of efficiently learning low-resolution features, thereby improving the learning ability of the student model and its performance.

[0091] In some embodiments, in order to further improve the performance of the student model, auxiliary tasks can be introduced into the teacher model and the student model respectively to prompt the model to learn more useful information at multiple levels. In the teacher model, auxiliary tasks can help it extract more diverse features, thereby enhancing its feature expression ability; while in the student model, auxiliary tasks help to improve its learning stability and generalization ability. Especially when learning complex tasks, these auxiliary tasks can not only enhance the model's sensitivity to different features, but also accelerate the training process and improve the accuracy and efficiency of the final model. By integrating auxiliary tasks into the training framework of knowledge distillation, the performance of the student model can be further improved while maintaining the distillation effect.

[0092] In order to introduce auxiliary tasks, before the first sample and the second sample are respectively input into the teacher model and the student model, it also includes: preprocessing the first sample and the second sample by specifying an enhancement operation to introduce self-supervised learning for the teacher model and the student model respectively based on the specified enhancement operation.

[0093] The electronic device can introduce new labels by performing a specified enhancement operation on the first sample and the second sample. For example, if the specified task is to rotate a specified angle, the first sample and the second sample after rotation will automatically be labeled with the rotation angle. In this way, training costs can be saved and existing labels in the data set can be fully utilized, even if auxiliary tasks are introduced in both the teacher model and the student model, without the need for additional manual annotation of the auxiliary task labels in the sample image.

[0094] Specifically, electronic devices can train these auxiliary tasks through self-supervised learning, that is, the auxiliary tasks are self-supervised auxiliary tasks, which can automatically obtain additional information from the enhanced operation without external annotation, thereby improving the learning efficiency and performance of the model. This self-supervised learning method can effectively reduce the cost of data annotation and enable the model to better learn the diverse features related to the task.

[0095] For example, the introduced auxiliary tasks (determined based on the specified enhancement operation, such as detecting the rotation angle) can be trained separately from the main task (target detection). Taking the detection of the rotation angle as an auxiliary task as an example, the teacher model and the student model can enhance the judgment and classification ability of the image subject by learning pictures at different angles, thereby helping the model learn more features or rules without affecting the training of the main task, thereby improving the expression ability of the learning model.

[0096] However, there are some problems with training the two tasks separately: on the one hand, the training process is relatively cumbersome; on the other hand, when the two tasks are trained separately, the later trained task may interfere with the knowledge learned in the previous task, resulting in reduced efficiency of knowledge transfer. For example, when the robot empty and robot loaded classification detection is the main task, and the rotation angle detection is the auxiliary task, if the auxiliary task is trained separately first, the model will mainly learn how to judge the rotation angle (such as 0° or 90°). Subsequently, in the classification task, the model will focus on learning how to distinguish between the robot empty and the robot loaded. The difference in knowledge in these two learning processes may lead to inefficient model learning because the learning objectives between different tasks are not effectively integrated.

[0097] Exemplarily, to solve this problem, the two tasks can be trained jointly, aggregating the main task and the auxiliary task into an aggregated task. In this way, the training process can form an end-to-end training framework, avoiding the cumbersome operations and inefficiency brought about by separate training. More importantly, the aggregation of tasks can expand the dimension of the training vector space, allowing the model to learn more features at the same time and improve its understanding of the main task and auxiliary tasks. This joint training method enables the two models to promote each other in the same training process, which not only improves the training efficiency, but also enhances the generalization ability of the model in a multi-task environment, thereby improving the final performance.

[0098] For example, see Figure 3 , Figure 3 A schematic diagram comparing the two training methods of separate training and joint training is shown. In separate training, the student model needs to imitate the prediction distribution of the teacher model in the main task (delivery item recognition) and the auxiliary task (rotation angle recognition) respectively. Specifically, the student model learns the output probability distribution of the teacher model in the target detection task in the main task, and learns the distribution of the teacher model in the rotation angle prediction task in the auxiliary task. Although this method can optimize each task separately, the lack of correlation between tasks may lead to low training efficiency and insufficient knowledge transfer between the two tasks.

[0099] In joint training, the student model only needs to simulate the distribution of the teacher model under the aggregated labels. That is, the student model learns the overall distribution of the teacher model by aggregating tasks (such as performing object detection and rotation angle prediction at the same time), rather than training each task separately. This method simplifies the training process by aggregating the main task and the auxiliary task into one task, and learns the features of multiple tasks at the same stage. For example, while learning the features of the main task for dishes, desserts, etc., it also learns the relevant features of different rotation angles. In this way, the student model can learn in a wider feature space, which improves the training efficiency and the overall performance of the model, while avoiding knowledge conflicts between tasks, making the knowledge transfer between tasks smoother.

[0100] Based on the aggregation task, the total loss also includes the aggregation task loss. For this reason, when calculating the total loss, the electronic device needs to calculate two losses based on the knowledge representation and the learning representation respectively. The electronic device calculates the distillation loss based on the knowledge representation and the learning representation of the teacher model and the student model under the aggregation task, and the loss reflects the difference between the two in feature learning. The electronic device calculates the aggregation task loss based on the predicted probability distribution of the aggregation task by the teacher model and the student model in the aggregation task, and the loss reflects the overall performance difference of the student model in the main task and the auxiliary task. The electronic device weightedly sums the two losses to obtain the total loss, and iteratively optimizes the student model so that it gradually learns the feature extraction ability and reasoning mode of the teacher model in the aggregation task, thereby improving the overall performance of the student model.

[0101] In some embodiments, preprocessing the first sample and the second sample based on a specified enhancement operation includes:

[0102] Step C1: The electronic device performs a set of specified enhancement operations on a first sample and a second sample respectively to obtain a first enhanced sample corresponding to the first sample and a second enhanced sample corresponding to the second sample.

[0103] In order to ensure the training effect of the auxiliary task, the electronic device can divide the same specified enhancement operation into different degrees, and perform it on the first sample and the second sample respectively, to obtain a group of first enhancement samples and a group of second enhancement samples respectively. These specified enhancement operations of different degrees can be regarded as a group of specified enhancement operations.

[0104] For example, in order to improve the accuracy of the robot's recognition of dishes in restaurant scenarios (the presence of steam will make the image blurry), the enhancement operation can be specified as blur processing, and specific processing of different intensities such as light blur, medium blur and heavy blur can be implemented. Light blur will maintain the overall structure and details of the image and reduce clarity; medium blur will blur the details, but the outline can still be recognized; heavy blur will greatly lose details and only retain the basic outline. Accordingly, for the first sample, it can obtain a first enhanced sample with a blur degree of light blur, medium blur and heavy blur; for the second sample, it can obtain a second enhanced sample with a blur degree of light blur, medium blur and heavy blur. Based on the two sets of enhanced samples for training, the teacher model and the student model can be exposed to a variety of image changes, so that the model can learn more robust features and discrimination capabilities, improve its adaptability to different changes and interferences, and enhance the generalization ability of the model.

[0105] Accordingly, if training is performed based on two enhanced samples, the knowledge representation also includes the first predicted label of the aggregated task, and the learning representation also includes the second predicted label of the aggregated task. Thus, the aggregated task loss can be calculated by the true label, the first predicted label, and the second predicted label of the aggregated task.

[0106] For example, suppose the true label of a training sample is (receipt, 90°), the first predicted label of the teacher model for the sample is (receipt, 90°), and the second predicted label of the student model is (receipt, 270°). By comparing the true label with the two predicted labels, the task loss can be calculated, and the student model can be optimized to make its prediction closer to the true label and the teacher model's prediction, thereby improving model performance.

[0107] In some embodiments, the specified enhancement operation includes at least one of a rotation operation, blur processing, and color perturbation. For example, rotation angle detection can help the model enhance its ability to judge and classify the subject of the image by learning images at different angles; blur processing can enable the model to adapt to the detection scene in the steam environment and improve its recognition ability under blurry or occluded conditions; color perturbation helps the model adapt to scenes with changing lighting or uneven lighting and improves the robustness of the model under changing lighting conditions. By introducing these specified enhancement operations, the model can improve the discrimination accuracy and adaptability in a variety of environments.

[0108] In some embodiments, see Figure 4 , the teacher model and the student model are both constructed using a three-stage convolutional neural network (CNN). If the CNN network is represented as a function f(·), it can be decomposed into the feature extraction part f e (·, w e ) and a linear classification head, where f c (·, w c ) represents the weight tensor. Consider X∈X as a mini-batch from the training dataset X, and assume that the vector space M={1,…,M} and the label space N={1,…N}, the aggregate vector space for this task is in represents the Cartesian product, the dimension of A is ||A||=M×N.

[0109] When X is transformed, we get is the feature embedding vector obtained by the feature extractor, where d is the embedding size. is the predicted probability distribution in the aggregation vector space A, is a normal-like probability distribution, where τ is a temperature hyperparameter used to measure the smoothness of the distribution.

[0110] Specifically, during the training of the teacher model, considering the need to train on the aggregate vector space, we use Instead of x as input, the superscript T indicates that x is the input of the teacher model. Therefore, the loss function constraining the convergence of teacher model training is expressed as:

[0111]

[0112] in, represents the true label; L CE represents the cross entropy loss.

[0113] In the process of knowledge distillation, As the input of the student model. The total loss of training the student model is decomposed into two parts: one is the loss between the predicted label and the true label of the aggregation task, and the other is the imitation loss between the hierarchical feature maps of the teacher model and the student model in the knowledge distillation process (the first distillation loss) and the imitation loss between the final logarithms (the second distillation loss), which are combined to form the distillation loss.

[0114] Aggregation task loss. Without considering knowledge distillation, the student model and the teacher model follow the same rules to train under the aggregation task. Therefore, the form of defining the aggregation task loss function is similar to that of the teacher model loss function, as follows:

[0115]

[0116] Distillation loss: The distillation loss can be further decomposed into the hierarchical feature map loss (i.e., the first distillation loss in the above example) and the final logarithmic loss (i.e., the second distillation loss in the above example) For the former, we assume that K intermediate layers in both the teacher model and the student model are sampled for knowledge distillation. These intermediate layers are denoted as This can be done using the KL divergence D KL Quantitatively measured, the first distillation loss formula is as follows:

[0117]

[0118] The formula for the second distillation loss is as follows:

[0119]

[0120] Finally, the aggregation task loss and distillation loss are combined to obtain the total loss L for student model training. S , as shown below, where λ1 and λ2 are hyperparameters used to balance different losses.

[0121]

[0122] In some embodiments, see Table 1 below, which shows The performance of the student model trained under different transformations is shown in Table 1. In the experimental setting, the teacher model is WRN40-2 and the student model is WRN40-1. Considering that the concept of cross-scale image processing is designed for the teacher model, different teacher processing methods are tried, while the processing method of the student model is always set to rotation.

[0123] Table 1

[0124]

[0125] In the above table, 4x means that the length and width of the original input image are enlarged to 4 times the original size through linear interpolation. Rot. means that {0°, 90°, 180°, 270°} rotation will be used as a self-supervised task in the aggregation task, and Perm. means that the image channels will be arranged in different orders {RGB, RBG, BRG, BGR, GRB, GBR}.

[0126] Based on the data in Table 1, it is found that the introduced auxiliary tasks The performance of the student model is improved to varying degrees. In addition, expanding the input scale of the teacher model will help improve the performance of the student model. In order to balance the performance improvement of the student model and the time and space cost of training, it is preferred to select the auxiliary task and expand the input scale of the teacher model to twice the original size.

[0127] In some embodiments, see Table 2 below, the Top-1 accuracy (%) of CIFAR100 by different SOTA methods for knowledge distillation between similar architectures. The bold and underlined numbers represent the best and suboptimal results. Obviously, the knowledge distillation method of the present application is significantly better than similar knowledge distillation methods.

[0128] Table 2

[0129]

[0130] In some embodiments, the effect in a small sample scenario. By retaining 25%, 50% and 75% of the training samples, the knowledge distillation method of the present application is compared with KD, CRD, SSKD and HSAKD in a small sample scenario, as shown in Table 4. It is worth noting that in the small sample scenario of 25%, the student model trained by the knowledge distillation method of the present application can achieve an accuracy of 70.50%. In a 100% data training scenario, the student model trained by traditional KD can only achieve a slightly higher accuracy of 70.66%. However, after introducing rotation as an auxiliary task, it can not only enrich the data set in scenarios with insufficient samples, but also the cross-scale images in the teacher model have finer-grained features, which can better guide the student model to learn, which makes the knowledge distillation method of the present application significantly different and superior to other knowledge distillation methods.

[0131] See Table 4 for a comparison of the Top-1 accuracy (%) of CIFAR100 in the few-shot scenario with different training sample ratios. The table shows the accuracy of the student model when ResNet56 and ResNet20 are used as the teacher-student pair.

[0132] Table 4

[0133]

[0134] The above experiments show that after the data set is transferred, the student model trained with the knowledge distillation method of this application outperforms all other mainstream methods. This result shows to some extent that the knowledge distillation method of this application can enable the model to learn knowledge that is not limited to the images in the training data set. The use of cross-scale image processing and self-supervised auxiliary tasks allows the model to focus on more semantic features during the learning process, thereby enabling the model to obtain better generalization performance on other data sets.

[0135] In some embodiments, in order to reduce the amount of robot calculation and computational complexity, the robot may deploy the converged student model obtained by the knowledge distillation method of the detection model in the above embodiment. Figure 5 , this application proposes a target detection method:

[0136] Step 510: During the execution of the task, the robot acquires a detection image.

[0137] Step 520: The robot performs target detection on the detection image through the student detection model to obtain a target detection result.

[0138] Step 530: The robot controls its own operation based on the target detection result and the current task status of the robot.

[0139] The student detection model deployed by the robot is used to perform target detection on the detection images obtained during the robot's task execution to obtain the corresponding target detection results. Based on the target detection results, the robot can control its own operation in combination with the current task status to complete the task.

[0140] For example, take the robot delivering food in a restaurant scenario. In the relevant scenario, if the robot receives a food delivery task, it can first move to the food pickup area to wait for food pickup. While waiting for food pickup, the robot can collect detection images through the camera for target detection, that is, food detection, and use the student model for detection. When the detection result is food, the robot can determine the next operation based on the current task state of waiting for food pickup and the detection result of food, that is, moving from the food pickup area to the delivery point.

[0141] For another example, the robot can use the student model to perform target detection, i.e., food identification, by collecting detection images through the camera, and recognize the name of the food. When it arrives at the delivery destination, it can announce the name of the dish and remind people to pick up the food. At the same time, the student model can be used for wrong picking identification, and when an error occurs, a wrong picking reminder will be given. When the student model detects that the food has been successfully picked up, the robot can automatically perform the next task.

[0142] The student model can also be used to detect receipt retention, taking the receipt as the target (whether the receipt is retained at the delivery target point, and if so, a reminder to take it away), return tray detection (whether there are dirty trays on the carrying device that need to be cleaned in the cleaning area, and if so, the robot navigates to the cleaning area), etc.

[0143] Whether you need to identify the presence of an object or the type of a specific object, you can set it up as needed, combine it with the robot's task status, control the robot's operation, and improve the robot's utilization rate and automation level.

[0144] The advantages of deploying student detection models on robots are mainly reflected in their lightweight and high efficiency. Compared with the teacher model, the student model is smaller and has lower computational complexity, making it suitable for running in resource-constrained environments. Especially in mobile robots, due to the fast reasoning speed of the student detection model, the robot can process and analyze the acquired detection images in real time and make decisions quickly, thereby improving the response speed of task execution. In addition, the student detection model maintains similar performance to the teacher model through knowledge distillation, while significantly reducing computational and memory consumption, improving the operating efficiency of the system.

[0145] In addition, the student model can dynamically adjust the robot's behavior based on the target detection results and the current task status to achieve more efficient task execution. Since the student model has low resource requirements, it can be deployed on the robot to reduce the hardware cost required by the robot while maintaining high task processing capabilities and robustness.

[0146] In some embodiments, if the auxiliary task is set to rotation angle detection, the student detection model only detects the target after deployment, without detecting the rotation angle of the detection image. If the auxiliary task is deblurring or color interference removal, the student model can not only perform target detection after deployment, but also perform deblurring or color interference removal operations in the image preprocessing stage, thereby improving image quality. This is especially important for food delivery robots because they usually need to work in complex environments and may encounter problems such as image blur or color distortion. By introducing auxiliary tasks, the robot can effectively handle these challenges while detecting targets, improving the accuracy and robustness of task execution.

[0147] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0148] Corresponding to the knowledge distillation method of the detection model in the above embodiment, Figure 6 A structural block diagram of the knowledge distillation device 6 of the detection model provided in the embodiment of the present application is shown. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0149] Reference Figure 6 The knowledge distillation device 6 of the detection model includes: the detection model includes a student model; the knowledge distillation device includes:

[0150] The following modules are triggered cyclically until a converged student model is obtained:

[0151] A training module 61 is used to input the first sample and the second sample into the teacher model and the student model for training, respectively, to obtain the knowledge representation of the teacher model and the learning representation of the student model; the first sample is obtained by scaling the second sample;

[0152] A calculation module 62, for calculating a distillation loss based on the knowledge representation and the learning representation;

[0153] The optimization module 63 is used to optimize the iterative student model based on the distillation loss.

[0154] Optionally, the knowledge representation includes a first intermediate layer feature of the teacher model and a first probability distribution of the target in the first sample; the learning representation includes a second intermediate layer feature of the student model and a second probability distribution of the target in the first sample; the total loss includes a distillation loss;

[0155] Accordingly, the calculation module 62 includes a first calculation unit, which is used to:

[0156] calculating a first distillation loss based on the first intermediate layer characteristic and the second intermediate layer characteristic;

[0157] A second distillation loss is calculated based on the first probability distribution and the second probability distribution.

[0158] Optionally, both the teacher model and the student model perform at least two stages of feature extraction; the student model is provided with a scaling module after each stage; the scaling module is used to align the scale of the feature map of the current stage of the student model with the scale of the feature map of the corresponding stage of the teacher model; the first intermediate layer feature includes a first feature map obtained after the feature extraction of the teacher model at each stage; the second intermediate layer feature includes a second feature map obtained after the feature extraction and scale scaling of the student model at each stage;

[0159] Accordingly, the first computing unit is specifically configured to:

[0160] At each stage, the corresponding stage loss is calculated based on the corresponding first feature map and the corresponding second feature map;

[0161] The first distillation loss was calculated based on the stage losses of each stage.

[0162] Optionally, the knowledge distillation device 6 of the detection model also includes a preprocessing module for:

[0163] Before the first sample and the second sample are respectively input into the teacher model and the student model, the first sample and the second sample are preprocessed by specifying an enhancement operation to introduce self-supervised learning into the teacher model and the student model respectively based on the specified enhancement operation; the teacher model and the student model both perform aggregation task detection;

[0164] The total loss also includes the aggregated task loss; accordingly, the calculation module 62 is specifically used for:

[0165] The distillation loss and aggregation task loss are calculated based on the knowledge representation and the learned representation.

[0166] Optionally, the preprocessing module is specifically used to: perform a set of specified enhancement operations on the first sample and the second sample respectively to obtain a first enhanced sample corresponding to the first sample and a second enhanced sample corresponding to the second sample;

[0167] Accordingly, the knowledge representation further includes a first prediction label, the learning representation further includes a second prediction label, and the computing module includes a second computing unit for:

[0168] The aggregation task loss is calculated based on the true label of the aggregation task, the first predicted label, and the second predicted label.

[0169] Optionally, the specified enhancement operation includes at least one of a rotation operation, a blurring process, and a color disturbance, and a group of specified enhancement operations includes specified enhancement operations of different degrees.

[0170] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0171] Figure 7 This is a schematic diagram of the physical structure of a robot provided in one embodiment of the present application. Figure 7 As shown, the robot 7 of this embodiment includes: at least one processor 70 ( Figure 7 Only one processor is shown in the figure), a memory 71, and a computer program 72 stored in the memory 71 and executable on at least one processor 70. When the processor 70 executes the computer program 72, the steps in the above-mentioned knowledge distillation method embodiment of any detection model are implemented, such as Figure 5 Steps 510-530 are shown.

[0172] The processor 70 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0173] In some embodiments, the memory 71 may be an internal storage unit of the robot 7, such as a hard disk or memory of the robot 7. In other embodiments, the memory 71 may also be an external storage device of the robot 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the robot 7.

[0174] Furthermore, the memory 71 may include both an internal storage unit and an external storage device of the robot 7. The memory 71 is used to store operating devices, applications, boot loaders, data, and other programs, such as program codes of computer programs, etc. The memory 71 may also be used to temporarily store data that has been output or is to be output.

[0175] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0176] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0177] An embodiment of the present application provides a computer program product. When the computer program product runs on a robot, the robot can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0178] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0179] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0180] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0181] In the embodiments provided in the present application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A knowledge distillation method for a detection model, characterized in that: The detection model includes a student model; and the knowledge distillation method includes: The following steps are executed repeatedly until the student model converges: Inputting the first sample and the second sample into the teacher model and the student model for training respectively, and obtaining the knowledge representation of the teacher model and the learning representation of the student model respectively; the first sample is obtained by scaling the second sample; Calculating a total loss based on the knowledge representation and the learned representation; The student model is iterated based on the total loss optimization.

2. The knowledge distillation method for the detection model according to claim 1, characterized in that: The knowledge representation includes a first intermediate layer feature of the teacher model and a first probability distribution of the target in the first sample; the learning representation includes a second intermediate layer feature of the student model and a second probability distribution of the target in the first sample; the total loss includes a distillation loss; Accordingly, the calculating the total loss based on the knowledge representation and the learning representation includes: calculating a first distillation loss based on the first intermediate layer characteristic and the second intermediate layer characteristic; A second distillation loss is calculated based on the first probability distribution and the second probability distribution.

3. The knowledge distillation method for the detection model according to claim 2, characterized in that: The teacher model and the student model both perform at least two stages of feature extraction; the student model is provided with a scaling module after each stage; the scaling module is used to align the scale of the feature map of the current stage of the student model with the scale of the feature map of the corresponding stage of the teacher model; The first intermediate layer features include a first feature map obtained by the teacher model after feature extraction at each stage; The second intermediate layer features include a second feature map obtained after feature extraction and scale scaling of the student model at each stage; Accordingly, the calculating the first distillation loss based on the first intermediate layer characteristic and the second intermediate layer characteristic includes: At each stage, the corresponding stage loss is calculated based on the corresponding first feature map and the corresponding second feature map; The first distillation loss is calculated based on the stage losses of each stage.

4. The knowledge distillation method for the detection model according to claim 2 or 3, characterized in that: Before inputting the first sample and the second sample into the teacher model and the student model respectively, the method further includes: Preprocessing the first sample and the second sample by specifying an enhancement operation to introduce self-supervised learning to the teacher model and the student model respectively based on the specified enhancement operation; both the teacher model and the student model perform aggregation task detection; The total loss also includes the aggregated task loss; accordingly, the total loss is calculated based on the knowledge representation and the learning representation, including: The distillation loss and the aggregation task loss are calculated based on the knowledge representation and the learned representation.

5. The knowledge distillation method for the detection model according to claim 4, characterized in that: The preprocessing of the first sample and the second sample based on the specified enhancement operation includes: Performing a set of the specified enhancement operations on the first sample and the second sample respectively to obtain a first enhanced sample corresponding to the first sample and a second enhanced sample corresponding to the second sample; Accordingly, the knowledge representation further includes a first prediction label of the aggregation task, the learning representation further includes a second prediction label of the aggregation task, and the inputting the first sample and the second sample into the teacher model and the student model, respectively, to obtain the knowledge representation of the teacher model and the learning representation of the student model, respectively, includes: Inputting the first enhanced sample and the second enhanced sample into the teacher model and the student model for training respectively, and obtaining the first predicted label and the second predicted label; The aggregate task loss is determined based on the following steps: The aggregate task loss is calculated based on the true label of the aggregate task, the first predicted label, and the second predicted label.

6. The knowledge distillation method for the detection model according to claim 5, characterized in that: The specified enhancement operation includes at least one of a rotation operation, a blurring process, and a color disturbance, and a group of the specified enhancement operations includes the specified enhancement operations at different degrees.

7. A target detection method, characterized in that: Applied to a robot, the robot is deployed with a trained student detection model obtained by the knowledge distillation method of the detection model according to any one of claims 1 to 6; the target detection method comprises: In the process of performing the task, the detection image is obtained; Performing target detection on the detection image by using the student detection model to obtain a target detection result; Based on the target detection result and the current task status of the robot, the robot is controlled to operate.

8. A knowledge distillation device for detecting a model, characterized in that: The detection model includes a student model; the knowledge distillation device: The following modules are triggered cyclically until the student model converges: A training module, used for inputting the first sample and the second sample into the teacher model and the student model for training respectively, so as to obtain the knowledge representation of the teacher model and the learning representation of the student model respectively; The first sample is obtained by scaling the second sample; A calculation module, configured to calculate a distillation loss based on the knowledge representation and the learning representation; An optimization module is used to optimize and iterate the student model based on the distillation loss.

9. A robot comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the knowledge distillation method of the detection model as described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by one or more processors, the knowledge distillation method of the detection model as described in any one of claims 1 to 7 is implemented.

Images