Detection model distillation method, target detection method, device and electronic equipment

Abstract

This application discloses a detection model distillation method, object detection method, apparatus, and electronic device, relating to the field of machine learning technology, for training a student model with higher accuracy to obtain better object detection results. The method includes: inputting sample images into a teacher model and a student model respectively; selecting positive teacher sample recognition results from the recognition results output by the teacher model after processing the sample images; for each positive teacher sample recognition result, determining a target anchor box matching the positive teacher sample recognition result from multiple preset anchor boxes after the student model processes the sample image; performing logical output distillation on each pair of matching positive teacher sample recognition results and target anchor boxes to obtain a logical output distillation loss; determining the detection task loss based on the ground truth of the recognition results; and training the student model based on the logical output distillation loss and the detection task loss to obtain the trained student model.

Description

Technical Field

[0001] This application relates to the field of machine learning technology, and in particular to a detection model distillation method, object detection method, apparatus and electronic equipment. Background Technology

[0002] The task of object detection is to identify objects in an image and determine their location and category. Object detection is typically achieved by training a detection model and then using that model to perform the detection function. However, models usually become more complex and computationally resource-intensive as their accuracy increases, making it difficult to store high-accuracy models on resource-constrained devices.

[0003] Knowledge distillation is a model compression method that uses a teacher model to induce the training of a student model. The pre-trained teacher model provides knowledge, and the simpler student model learns from the teacher model's knowledge through knowledge distillation training. This allows the knowledge of a complex model to be transferred to a model with fewer parameters and weaker learning ability. The student model retains the accuracy of the teacher model while meeting the storage and computation requirements of devices with limited computing resources.

[0004] However, in current methods of training models through knowledge distillation, the output of the teacher model is passed to the student model, which contains a lot of useless information. This limits the student model's ability to learn the knowledge of the teacher model, resulting in low accuracy of the trained student model. When using the student model for object detection, it cannot achieve good object detection results. Summary of the Invention

[0005] This application provides a detection model distillation method, target detection method, apparatus, and electronic device for training a more accurate student model, so as to obtain more accurate detection results when using the trained student model for target detection.

[0006] To achieve the above technical objectives, this application adopts the following technical solution:

[0007] In a first aspect, this application provides a method for detecting model distillation, comprising:

[0008] The training teacher model, sample images containing target objects, and ground truth values ​​of the target object recognition results corresponding to the sample images are obtained. The teacher model is used to output the recognition results of the target objects in the sample images.

[0009] The sample images are input into the teacher model and the student model respectively. The recognition results output by the teacher model after processing the sample images are filtered to obtain the positive sample recognition results of the teacher.

[0010] For each teacher positive sample recognition result, a target anchor box matching the teacher positive sample recognition result is determined from multiple preset anchor boxes after the student model processes the sample image;

[0011] Logical output distillation is performed on each matching combination to obtain the logical output distillation loss, which is used to indicate the degree of difference between the teacher model and the student model in the output layer. A matching combination includes a teacher positive sample recognition result and a target anchor box that matches the teacher positive sample recognition result.

[0012] Based on the true value of the recognition result, the detection task loss is determined. The detection task loss is used to indicate the degree of difference between the student model's recognition result of the target object and the true value of the recognition result.

[0013] The student model is trained using the logical output distillation loss and the detection task loss to obtain the trained student model.

[0014] The technical solution provided in this application brings at least the following beneficial effects: By filtering the recognition results output by the teacher model after processing the sample images, positive teacher sample recognition results are obtained, and useless information output by the teacher model is filtered out. Then, target anchor boxes matching each positive teacher sample recognition result are determined from multiple preset anchor boxes after the student model processes the sample images. When performing logical output distillation on the target anchor boxes and the positive teacher sample recognition results matching the target anchor boxes, the student model can learn the recognition results of the teacher model one-to-one and more specifically, without having to learn the useless recognition results output by the teacher model. The student model obtained based on this training process has higher accuracy and precision in target object detection, and can better meet the user's needs.

[0015] In one possible implementation, the teacher positive sample recognition result is the teacher positive sample bounding box; after processing the sample image from the student model, a target anchor box matching the teacher positive sample recognition result is determined from a set of preset anchor boxes, including: for each anchor box, determining the intersection-union ratio (IU) between the anchor box and the teacher positive sample bounding box; and determining the anchor box indicated by the maximum value in the IU as the target anchor box matching the teacher positive sample bounding box.

[0016] In one possible implementation, before performing logical output distillation on each matching combination to obtain the logical output distillation loss, the method further includes:

[0017] In one possible implementation, the teacher positive sample recognition result is corrected based on the ground truth of the recognition result to obtain the corrected teacher positive sample recognition result; the detection task loss is determined based on the ground truth of the recognition result, including: defining the area in the sample image that is marked by the corrected teacher positive sample recognition result but not marked by the ground truth of the recognition result as the non-processing area, the non-processing area is used to instruct the student model not to perform target object recognition in the non-processing area; the detection task loss is obtained based on the ground truth of the recognition result and the non-processing area in the sample image.

[0018] In one possible implementation, the teacher positive sample recognition result is the teacher positive sample bounding box and the corresponding teacher positive sample classification information, and the recognition result truth value is the recognition result truth value bounding box and the corresponding truth value classification information. Based on the recognition result truth value, the teacher positive sample recognition result is corrected to obtain the corrected teacher positive sample recognition result, including: determining the intersection-union ratio (IUR) between the recognition result truth value bounding box and the teacher positive sample bounding box; deleting teacher positive sample bounding boxes whose IUR is greater than a preset threshold and whose teacher positive sample classification information is different from the truth value classification information, thus obtaining the corrected teacher positive sample bounding box.

[0019] In one possible implementation, a target anchor box corresponds to a student prediction box and a student classification information, with the student prediction box obtained by the student model based on the target anchor box. Logical output distillation is performed on each matching combination to obtain the logical output distillation loss, including: for each matching combination, determining the localization loss between the student prediction box corresponding to the target anchor box and the corrected teacher positive sample box, and the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box; the localization loss indicates the degree of difference between the positions of the target objects identified by the teacher model and the student model, and the classification loss indicates the degree of difference between the output classification information of the teacher model and the student model; and the foreground / background confidence of the teacher model is obtained. The system uses background confidence information for both the teacher and student models. The teacher model's background represents the region output by the teacher model after processing the sample image, excluding the corrected teacher positive bounding box. The student model's background represents the region output by the student model after processing the sample image, excluding the student prediction box. Based on the teacher and student models' background confidence information, the system determines the confidence loss between their respective backgrounds. Finally, based on the localization loss, the classification loss between the student classification information corresponding to the target anchor box and the teacher positive bounding box corresponding to the corrected teacher positive bounding box, and the confidence loss between their respective backgrounds, the system determines the logical output distillation loss.

[0020] In one possible implementation, the feature layer of the student model includes a student backbone network and a student detection head, and the feature layer of the teacher model includes a teacher backbone network and a teacher detection head. The method further includes: when the teacher detection head and the student detection head have the same structure, performing feature distillation on the student backbone network using the teacher backbone network, and performing feature distillation on the student detection head using the teacher detection head, to obtain a feature distillation loss. The student backbone network is used to extract feature information from sample images and transmit it to the student detection head. The student detection head is used to detect target objects and their classification information from the feature information. The feature distillation loss is used to indicate the differences between the teacher model and the student detection head. The model exhibits varying degrees of difference at the feature layer. When the structures of the teacher and student detection heads differ, feature distillation is performed on the student backbone network using the teacher backbone network, and feature distillation is also performed on the auxiliary detection head using the teacher detection head, resulting in a feature distillation loss. The auxiliary detection head has the same structure as the teacher detection head, and the student backbone network is also used to transmit feature information to the auxiliary detection head. Based on the logical output distillation loss and the detection task loss, the student model is trained to obtain the trained student model. This includes training the student model based on the feature distillation loss, logical output distillation loss, and detection task loss to obtain the trained student model.

[0021] Secondly, this application provides a target detection method, including:

[0022] Acquire the image to be detected;

[0023] The image to be detected is input into the student model, and the target object is identified from the image. The student model is trained based on any of the detection model distillation methods shown in the first aspect.

[0024] Thirdly, this application provides a detection model distillation apparatus, comprising:

[0025] The acquisition module is used to acquire the trained teacher model, sample images containing target objects, and ground truth values ​​of the recognition results of the target objects corresponding to the sample images. The teacher model is used to output the recognition results of the target objects in the sample images.

[0026] Processing module, used for:

[0027] The sample images are input into the teacher model and the student model respectively. The recognition results output by the teacher model after processing the sample images are filtered to obtain the positive sample recognition results of the teacher.

[0028] For each teacher positive sample recognition result, a target anchor box matching the teacher positive sample recognition result is determined from multiple preset anchor boxes after the student model processes the sample image;

[0029] Logical output distillation is performed on each matching combination to obtain the logical output distillation loss, which is used to indicate the degree of difference between the teacher model and the student model in the output layer. A matching combination includes a teacher positive sample recognition result and a target anchor box that matches the teacher positive sample recognition result.

[0030] Based on the true value of the recognition result, the detection task loss is determined. The detection task loss is used to indicate the degree of difference between the student model's recognition result of the target object and the true value of the recognition result.

[0031] The student model is trained using the logical output distillation loss and the detection task loss to obtain the trained student model.

[0032] In one possible implementation, the teacher positive sample identification result is the teacher positive sample bounding box; the processing module is specifically used to: for each anchor box, determine the intersection-union ratio (IUR) between the anchor box and the teacher positive sample bounding box; and determine the anchor box indicated by the maximum value in the IUR as the target anchor box that matches the teacher positive sample bounding box.

[0033] In one possible implementation, the processing module is further configured to: correct the teacher positive sample recognition result based on the ground truth of the recognition result to obtain the corrected teacher positive sample recognition result; specifically, the processing module is configured to: identify the area in the sample image that is marked by the modified teacher positive sample recognition result and not marked by the ground truth of the recognition result as the non-processing area, the non-processing area is used to instruct the student model not to perform target object recognition in the non-processing area; and obtain the detection task loss based on the ground truth of the recognition result and the non-processing area in the sample image.

[0034] In one possible implementation, the teacher positive sample recognition result is the teacher positive sample box and the teacher positive sample classification information corresponding to the teacher positive sample box, and the recognition result truth value is the recognition result truth value box and the truth value classification information corresponding to the recognition result truth value box; the processing module is specifically used to: determine the intersection-union ratio between the recognition result truth value box and the teacher positive sample box; delete the teacher positive sample boxes that have an intersection-union ratio greater than a preset threshold and whose teacher positive sample classification information is different from the truth value classification information, and obtain the corrected teacher positive sample box.

[0035] In one possible implementation, a target anchor box corresponds to a student prediction box and student classification information. The student prediction box is obtained by the student model based on the target anchor box. The processing module is specifically used to: for each matching combination, determine the localization loss between the student prediction box corresponding to the target anchor box and the corrected teacher positive sample box, and the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box; the localization loss indicates the degree of difference between the positions of the target objects identified by the teacher model and the student model, and the classification loss indicates the degree of difference between the classification information output by the teacher model and the student model; obtain the foreground and background confidence information of the teacher model, and the student model... The foreground and background confidence information is used to determine the confidence loss between the foreground and background of the teacher model and the student model. The foreground and background of the teacher model represent the region output by the teacher model after processing the sample image, excluding the corrected teacher positive sample box. The foreground and background of the student model represent the region output by the student model after processing the sample image, excluding the student prediction box. Based on the foreground and background confidence information of the teacher model and the student model, the confidence loss between the foreground and background of the teacher model is determined. Based on the localization loss, the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box, and the confidence loss between the foreground and background of the teacher model and the student model, the logical output distillation loss is determined.

[0036] In one possible implementation, the feature layer of the student model includes a student backbone network and a student detection head, and the feature layer of the teacher model includes a teacher backbone network and a teacher detection head. The processing module is further configured to: when the teacher detection head and student detection head have the same structure, perform feature distillation on the student backbone network using the teacher backbone network, and perform feature distillation on the student detection head using the teacher detection head, to obtain a feature distillation loss. The student backbone network is used to extract feature information from sample images and transmit it to the student detection head, which is used to detect target objects and their classification information from the feature information. The feature distillation loss is used to indicate the degree of difference between the teacher model and the student model in the feature layer. When the teacher detection head and student detection head have different structures, perform feature distillation on the student backbone network using the teacher backbone network, and perform feature distillation on the auxiliary detection head using the teacher detection head, to obtain a feature distillation loss. The auxiliary detection head has the same structure as the teacher detection head, and the student backbone network is also used to transmit feature information to the auxiliary detection head. Specifically, the processing module is used to train the student model based on the feature distillation loss, the logical output distillation loss, and the detection task loss, to obtain the trained student model.

[0037] Fourthly, this application provides a target detection device, comprising:

[0038] The acquisition module is used to acquire the image to be detected;

[0039] The processing module is used to input the image to be detected into the student model and identify the target object from the image to be detected. The student model is trained based on any of the detection model distillation methods shown in the first aspect.

[0040] Fifthly, this application provides an electronic device, comprising: one or more processors; one or more memories; wherein the one or more memories are used to store computer program code, the computer program code including computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes any of the detection model distillation methods provided in the first aspect above, or any of the target detection methods provided in the second aspect.

[0041] Sixthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed on a computer, cause the computer to perform any of the detection model distillation methods provided in the first aspect or any of the target detection methods provided in the second aspect.

[0042] Seventhly, this application provides a computer program product including computer instructions that, when executed on an electronic device, cause the electronic device to perform any of the detection model distillation methods provided in the first aspect or any of the target detection methods provided in the second aspect.

[0043] For a detailed description of aspects two through seven and their various implementations in this application, please refer to the detailed description in aspect one and its various implementations; and for a detailed description of the beneficial effects of aspects two through seven and their various implementations, please refer to the beneficial effect analysis in aspect one and its various implementations, which will not be repeated here.

[0044] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of a model distillation system applicable to a detection model distillation method provided in an embodiment of this application;

[0046] Figure 2 This is a schematic diagram of the structure of a detection model distillation apparatus provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0048] Figure 4 A flowchart of a detection model distillation method provided in this application embodiment;

[0049] Figure 5 This application provides an illustration of an application scenario for a detection model distillation method. Figure 1 ;

[0050] Figure 6 A logical schematic diagram of a detection model distillation method provided in an embodiment of this application;

[0051] Figure 7 This application provides an illustration of an application scenario for a detection model distillation method. Figure 2 ;

[0052] Figure 8 This application provides an illustration of an application scenario for a detection model distillation method. Figure 3 ;

[0053] Figure 9 A schematic diagram of the structure of another detection model distillation apparatus provided in this application embodiment;

[0054] Figure 10 This is a schematic diagram of the structure of a target detection device provided in an embodiment of this application;

[0055] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0057] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0058] The task of object detection is to identify objects in an image and determine their location and category. Object detection is typically achieved by training a detection model and then using that model to perform the detection function. However, models usually become more complex and computationally resource-intensive as their accuracy increases, making it difficult to store high-accuracy models on resource-constrained devices.

[0059] Knowledge distillation is a model compression method that uses a teacher model to induce the training of a student model. The pre-trained teacher model provides knowledge, and the simpler student model learns from the teacher model's knowledge through knowledge distillation training. This allows the knowledge of a complex model to be transferred to a model with fewer parameters and weaker learning ability. The student model retains the accuracy of the teacher model while meeting the storage and computation requirements of devices with limited computing resources.

[0060] However, in current methods of training models through knowledge distillation, the output of the teacher model is passed to the student model, which contains a lot of useless information. This limits the student model's ability to learn the knowledge of the teacher model, resulting in low accuracy of the trained student model. When using the student model for object detection, it cannot achieve good object detection results.

[0061] In response, this application provides a detection model distillation method. By filtering the recognition results output by the teacher model after processing the sample image, positive teacher sample recognition results are obtained, and useless information output by the teacher model is filtered out. Then, target anchor boxes matching each positive teacher sample recognition result are determined from multiple preset anchor boxes after the student model processes the sample image. When performing logical output distillation on the target anchor boxes and the positive teacher sample recognition results matching the target anchor boxes, the student model can learn the recognition results of the teacher model one-to-one and more specifically, without having to learn the useless recognition results output by the teacher model. The student model obtained based on this training process has higher accuracy and precision in target object detection, which better meets the user's needs.

[0062] Figure 1 A schematic diagram of the model distillation system to which the detection model distillation method provided in this application is applicable is shown. For example... Figure 1 As shown, the model distillation system 1 includes a model distillation detection device 10 and a target detection device 20.

[0063] In this process, a communication connection is established between the detection model distillation device 10 and the target detection device 20. It should be understood that the connection method can be a wireless connection, such as a Bluetooth connection or a wireless fidelity (Wi-Fi) connection; or, the connection method can be a wired connection, such as a fiber optic connection, etc., and there is no limitation on this.

[0064] In some embodiments, the detection model distillation apparatus 10 is used to perform knowledge distillation training and ground value training on the student model based on the trained teacher model, sample images containing target objects, and the ground values ​​of the recognition results of the target objects corresponding to the sample images, so that the trained student model can achieve the accuracy of the teacher model in detecting target objects.

[0065] In some embodiments, the knowledge distillation training process of the student model by the detection model distillation apparatus 10 includes feature distillation and logic output distillation. Based on this, the detection model distillation apparatus 10 can include, for example... Figure 2 As shown, the system includes a filtering module, a correction module, a matching module, and a logical output distillation module. The filtering module filters the recognition results output by the teacher model after processing the sample images. The correction module corrects the positive teacher sample recognition results output by the teacher model during ground truth training. The matching module matches the positive teacher sample recognition results output by the teacher model with the preset anchor boxes of the student model. The logical output distillation module performs logical output distillation on both the teacher and student models.

[0066] In some embodiments, the detection model distillation device 10 can be connected to an image acquisition device to obtain an image containing the target object acquired by the image acquisition device. Then, the user can identify the target object in the image containing the target object, and the detection model distillation device 10 determines the image calibrated by the user as a sample image.

[0067] In some embodiments, the detection model distillation apparatus 10 is connected to a terminal device used by a user, and receives a sample image containing a target object sent by the terminal device used by the user to the detection model distillation apparatus 10, as well as the true value of the recognition result of the target object corresponding to the sample image.

[0068] In some embodiments, the target detection device 20 is used to run a trained student model, process the image to be detected, and output the target detection result.

[0069] In practical applications, the target detection device 20 is usually an embedded edge computing device, which is a computing device with small size, low power consumption and high performance. Therefore, the embedded edge computing device cannot run high-precision and complex teacher models. It is necessary to deploy the trained and relatively simple student models in the embedded edge computing device to meet the computing requirements of the embedded edge computing device for target detection.

[0070] In some embodiments, the detection model distillation apparatus 10 may be an electronic device with data processing capabilities, or a functional module within that electronic device, without limitation. For example, the electronic device may be a server, which can be a single server or a server cluster composed of multiple servers. As another example, the electronic device may be a mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, and other terminal devices. This application does not impose any special limitations on the specific form of the electronic device.

[0071] In some embodiments, the detection model distillation apparatus 10 and the target detection apparatus 20 can be as follows: Figure 1 The device shown is a standalone unit; alternatively, the detection model distillation device 10 and the target detection device 20 can be integrated together.

[0072] In some embodiments, the model distillation system 1 may include multiple target detection devices 20, and the user can deploy the student model trained by the detection model distillation device 10 on the multiple target detection devices 20.

[0073] The hardware structure of the aforementioned detection model distillation apparatus 10 includes: Figure 3 The components included in the computing device shown. The following are examples... Figure 3 Taking the computing device shown as an example, the hardware structure of the detection model distillation device 10 is introduced.

[0074] like Figure 3 As shown, the computing device may include a processor 301, a memory 302, a communication interface 303, and a bus 304. The processor 301, the memory 302, and the communication interface 303 can be connected via the bus 304.

[0075] Processor 301 is the control center of the computing device. It can be a single processor or a collective term for multiple processing elements. For example, processor 301 can be a general-purpose central processing unit (CPU) or other general-purpose processors. Among them, the general-purpose processor can be a microprocessor or any conventional processor.

[0076] As one embodiment, processor 301 may include one or more CPUs, for example Figure 3 CPU 0 and CPU 1 are shown in the diagram.

[0077] The memory 302 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0078] In one possible implementation, the memory 302 can exist independently of the processor 301. The memory 302 can be connected to the processor 301 via a bus 304 and is used to store instructions or program code. When the processor 301 calls and executes the instructions or program code stored in the memory 302, it can implement the model deployment method provided in the embodiments of this application.

[0079] In another possible implementation, the memory 302 can also be integrated with the processor 301.

[0080] Communication interface 303 is used for connecting the computing device to other devices via a communication network, such as Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. Communication interface 303 may include a receiving unit for receiving data and a transmitting unit for transmitting data.

[0081] Bus 304 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0082] It should be pointed out that, Figure 3 The structure shown does not constitute a limitation on the computing device, except Figure 3 In addition to the components shown, the computing device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0083] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0084] like Figure 4 As shown, this application embodiment provides a detection model distillation method, which can be executed by the above-described detection model distillation apparatus 10. The method includes the following steps:

[0085] S101. Obtain the trained teacher model, sample images containing the target object, and the ground truth values ​​of the target object recognition results corresponding to the sample images.

[0086] The teacher model is used to output the recognition results of target objects in the sample images.

[0087] For example, the sample image containing the target object can be an image of the target object captured by a camera, or an image of the target location or target application scenario transmitted by the user to the detection model distillation device via a mobile phone, computer, or other terminal device. For instance, if the target object is a license plate, the sample image containing the target object can be an image of a vehicle on the road captured by a camera installed on the road, or an image of a vehicle in the target parking lot transmitted by the user to the detection model distillation device via a computer.

[0088] S102. Input the sample images into the teacher model and the student model respectively. Filter the recognition results output by the teacher model after processing the sample images to obtain the positive sample recognition results of the teacher.

[0089] The detection model distillation device can filter the recognition results output by the teacher model after processing the sample images through non-maximum suppression (NMS) operation to obtain the positive sample recognition results of the teacher.

[0090] In some embodiments, the teacher positive sample recognition result is the teacher positive sample bounding box and the teacher positive sample classification information corresponding to the teacher positive sample bounding box.

[0091] For example, such as Figure 5 As shown, the target object is a vehicle. After inputting a sample image containing a vehicle into the teacher model, the area enclosed by two boxes in the recognized image output by the teacher model is the teacher positive sample box output by the teacher model after processing the sample image. The teacher positive sample classification information of the teacher positive sample box is also marked, which is "vehicle" marked in the figure.

[0092] In some embodiments, after the sample images are input into the teacher model and the student model respectively, the feature layers of the teacher model and the student model first process the sample images, extract feature information from the sample images, and then transmit the feature information to the output layers of the teacher model and the student model respectively to obtain the recognition results of the sample images.

[0093] In some embodiments, the feature layer of the student model includes a student backbone network and a student detection head, and the feature layer of the teacher model includes a teacher backbone network and a teacher detection head. When the teacher detection head and the student detection head have the same structure, the detection model distillation device performs feature distillation on the student backbone network using the teacher backbone network and on the student detection head using the teacher detection head, obtaining a feature distillation loss. When the teacher detection head and the student detection head have different structures, the teacher backbone network performs feature distillation on the student backbone network and on the auxiliary detection head using the teacher detection head, obtaining a feature distillation loss. The student backbone network is used to extract feature information from sample images and transmit it to the student detection head. The student detection head is used to detect target objects and their classification information from the feature information. The auxiliary detection head has the same structure as the teacher detection head, and the student backbone network is also used to transmit feature information to the auxiliary detection head. The feature distillation loss indicates the degree of difference between the teacher model and the student model at the feature layer.

[0094] For example, such as Figure 6 As shown, after inputting sample images containing the target object into the teacher model and student model respectively, on the teacher model side, the sample images sequentially pass through the teacher backbone network and teacher detection head of the teacher model's feature layer; on the student model side, the sample images sequentially pass through the student backbone network and student detection head of the student model's feature layer. The detection model distillation device first aligns the dimensions of the teacher model and student model respectively through an adaptive layer (which can also be a fully connected layer or a 1×1 convolutional layer), and then uses the mean squared error (MSE) loss, the specific calculation formula of which is... Feature distillation learning is performed on the teacher and student models at each level. The specific feature distillation loss formula can be shown below:

[0095]

[0096] Among them, L feats This represents the characteristic distillation loss, where l represents the layer. This represents the characteristics of teacher layer l. φ represents the characteristics of the student layer l, and φ represents the adaptive layer.

[0097] Specifically, when the structures of the teacher detection head and the student detection head are different, an auxiliary detection head with the same structure as the teacher detection head is added to the student model. The sample image passes through the student backbone network and the auxiliary detection head in sequence. Based on the above-mentioned calculation method of feature distillation loss, the feature distillation loss is determined.

[0098] In this way, by adding an auxiliary detector with the same structure as the teacher detector in the student model when the structures of the teacher detector and the student detector are different, the feature distillation process can be adapted to teacher and student models with the same structure, as well as teacher and student models with different structures. The student model can learn the knowledge of different teacher models, and the application scope is wider.

[0099] In some embodiments, after obtaining the teacher positive sample identification result, the detection model distillation device can also correct the teacher positive sample identification result based on the true value of the identification result to obtain the corrected teacher positive sample identification result.

[0100] Specifically, the teacher positive sample recognition result consists of the teacher positive sample bounding box and the corresponding teacher positive sample classification information. The recognition result ground truth value is the recognition result ground truth bounding box and the corresponding ground truth classification information. The detection model distillation device determines the intersection-union ratio (IUR) between the recognition result ground truth bounding box and the teacher positive sample bounding box; it deletes teacher positive sample bounding boxes whose IUR is greater than a preset threshold and whose teacher positive sample classification information differs from the ground truth classification information, thus obtaining the corrected teacher positive sample bounding box.

[0101] For example, suppose a user marks 5 ground truth boxes for recognition results in a sample image. The teacher model outputs 5 positive teacher boxes for the sample image. The intersection-union ratio (IUU) of these 5 positive teacher boxes with their respective ground truth boxes is greater than a preset threshold. If the classification information corresponding to one of the positive teacher boxes is "car" and the classification information corresponding to the ground truth box of the same positive teacher box is "bus", then the positive teacher box is deleted, and the remaining 4 positive teacher boxes are the corrected positive teacher boxes.

[0102] In this way, the teacher's positive sample boxes are corrected by recognizing the true values ​​of the results. As a result, the student model can achieve better learning results when learning the corrected teacher's positive sample boxes, and the student model training results can also be better.

[0103] S103. For each teacher positive sample recognition result, determine the target anchor box that matches the teacher positive sample recognition result from multiple preset anchor boxes after processing the sample image from the student model.

[0104] For each anchor point in the sample image, the student model generates a preset number of anchor boxes corresponding to that anchor point based on multiple pre-set dimensions. In this embodiment, the preset number of anchor boxes after the student model processes the sample image refers to each anchor box corresponding to all anchor points in the sample image. Anchor points can be pixels in the sample image.

[0105] In some embodiments, for each anchor frame, the detection model distillation apparatus determines the intersection-union ratio (IU) between the anchor frame and the teacher positive sample frame; the anchor frame indicated by the maximum value in the IU is determined as the target anchor frame that matches the teacher positive sample frame.

[0106] The intersection-union ratio of the anchor frame and the teacher's positive sample frame can be the ratio of the area of ​​the intersection and the area of ​​the union of the anchor frame's region and the teacher's positive sample frame's region.

[0107] For example, based on Figure 5 ,like Figure 7 As shown, in the processing results of the sample image output by the student model, each pixel is an anchor point. Taking the labeled anchor point 'a' as an example, if the preset number is 6, the dashed box represents the 6 anchor boxes of anchor point A. Taking the teacher's positive sample box A as an example, the intersection-union ratio (IUR) of each of the 6 anchor boxes of each anchor point with the teacher's positive sample box A is calculated. If the IUR of anchor point 'a' with anchor box 3 of all anchor points is the largest, then anchor point 3 is the target anchor box that matches the teacher's positive sample box A.

[0108] In this way, by determining the target anchor box that matches each teacher's positive sample box, the student model can learn the knowledge of the teacher model more effectively during subsequent logic output distillation.

[0109] S104. Perform logical output distillation on each matching combination to obtain the logical output distillation loss.

[0110] The logical output distillation loss is used to indicate the degree of difference between the teacher model and the student model in the output layer; a matching combination includes a teacher positive sample identification result and a target anchor box that matches the teacher positive sample identification result.

[0111] In some embodiments, a target anchor box corresponds to a student prediction box and student classification information, and the student prediction box is obtained by the student model based on the target anchor box. Step S104 can be specifically implemented as follows: For each matching combination, the detection model distillation device determines the localization loss between the student prediction box corresponding to the target anchor box and the corrected teacher positive sample box, and the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box; the detection model distillation device obtains the foreground and background confidence information of the teacher model and the student model, and determines the confidence loss between the foreground and background of the teacher model and the student model based on the foreground and background confidence information of the teacher model and the student model; then the detection model distillation device determines the logic output distillation loss based on the localization loss, the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box, and the confidence loss between the foreground and background of the teacher model and the student model. Specifically, the localization loss indicates the difference between the positions of the target objects identified by the teacher model and the student model, the classification loss indicates the difference between the classification information output by the teacher model and the student model, the foreground / background of the teacher model represents the region output by the teacher model after processing the sample image, excluding the corrected teacher positive bounding box, and the foreground / background of the student model represents the region output by the student model after processing the sample image, excluding the student predicted bounding box. The student predicted bounding box represents the target object recognition result of the student model outputting the sample image.

[0112] Generally, the confidence level of points in the recognition results of the target object can be set to 1, and the confidence level of points in the foreground and background can be set to 0.

[0113] Specifically, after the teacher positive sample identification result is corrected, each matching combination includes a corrected teacher positive sample bounding box and a target anchor box that matches the teacher positive sample bounding box.

[0114] For example, for a target anchor box and a teacher positive sample box in a matching combination, the localization loss between the student prediction box corresponding to the target anchor box and the teacher positive sample box can be obtained by calculating the IoU loss. The classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the teacher positive sample box can be obtained by calculating the CE loss or KL loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the teacher positive sample box. Furthermore, the confidence loss between the foreground and background of the teacher model and the foreground and background of the student model can also be obtained by calculating the CE loss or KL loss between the foreground and background confidence of the teacher model and the foreground and background of the student model. The formula for calculating the IoU loss is L... IoU= -ln(IoU), the formula for calculating CE loss is: The formula for calculating KL loss is as follows:

[0115] Based on this, the formula for calculating the distillation loss of the logic output can be as follows:

[0116] L logits =L reg +L cls +L conf

[0117] Among them, L logits L represents the logic output distillation loss. reg L represents the localization loss between the student's predicted bounding box and the teacher's positive bounding box corresponding to the target anchor box. cls L represents the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the teacher positive sample box. conf This represents the confidence loss between the context of the teacher model and the context of the student model.

[0118] In this way, by performing logical output distillation on the matching teacher positive sample boxes and target anchor boxes, the teacher output results are divided into positive sample boxes and foreground and background (excluding positive sample boxes) for distillation learning. This not only allows the student model to learn the detection results of the teacher model on the target object one-to-one, but also to learn the detection results of the teacher model on the background information in the sample image. Based on this, the training effect of the student model is also better.

[0119] S105. Based on the true value of the recognition result, determine the loss of the detection task.

[0120] The detection task loss is used to indicate the degree of difference between the student model's recognition result of the target object and the true value of the recognition result.

[0121] In some embodiments, step S105 can be specifically implemented as follows: the detection model distillation device determines the region in the sample image that is marked by the corrected teacher positive sample recognition result and not marked by the true value of the recognition result as the non-processing region; based on the true value of the recognition result and the non-processing region in the sample image, the detection task loss is obtained. The non-processing region is used to instruct the student model not to perform target object recognition in the non-processing region.

[0122] For example, such as Figure 8As shown, the areas marked with solid lines in the sample image are the corrected teacher positive sample boxes, and the areas marked with dashed lines are the ground truth boxes of the recognition results. Therefore, the areas marked with ground truth boxes of the recognition results are taken as the real recognition results corresponding to the target object. The areas where neither dashed nor solid lines are marked are determined as the foreground and background, and the areas marked with solid lines but not with dashed lines are determined as unprocessed areas. Thus, when training the student model with ground truth, only the real recognition results and the foreground and background are processed to determine the detection task loss.

[0123] In this way, when training the student model with ground truth, the unprocessed areas are removed. That is, the student model does not process the areas in the sample image that are marked by the teacher's positive sample recognition results and whose recognition results are not marked with ground truth. The student model will not recognize the unprocessed areas as background outside the target object. This can avoid the situation where the logic output distillation and ground truth training conflict due to the teacher model recognizing the target object because of human omission of annotation.

[0124] S106. Based on the logic output distillation loss and the detection task loss, the student model is trained to obtain the trained student model.

[0125] In some embodiments, step S106 can be specifically implemented as follows: training the student model based on feature distillation loss, logic output distillation loss, and detection task loss to obtain the trained student model.

[0126] The detection model distillation device modifies the parameters of the student model based on feature distillation loss, logic output distillation loss, and detection task loss. It continuously inputs different sample images and modifies the parameters of the student model. When the student model's recognition result of the target object reaches the expected level, the training process of the student model ends, thus completing the training process of the student model.

[0127] In this way, the final trained student model not only learns the true value of the recognition result, but also learns the teacher model's ability to detect target objects, resulting in a higher accuracy of the final trained student model.

[0128] from Figure 5 As can be seen, by correcting the teacher positive sample identification results involved in the logic output distillation process through the identification result truth value, the teacher positive sample identification results also correct the truth value training process, and finally a better student model training result can be obtained.

[0129] Figure 4The technical solution shown brings at least the following beneficial effects: By filtering the recognition results output by the teacher model after processing the sample images, positive teacher sample recognition results are obtained, and useless information output by the teacher model is filtered out. Then, target anchor boxes matching each positive teacher sample recognition result are determined from multiple anchor boxes preset by the student model. When performing logical output distillation on the target anchor boxes and the positive teacher sample recognition results matching the target anchor boxes, the student model can learn the recognition results of the teacher model one-to-one and more specifically, without having to learn the useless recognition results output by the teacher model. The student model obtained based on this training process has higher accuracy and precision in target object detection, and can better meet the user's needs.

[0130] In some embodiments, this application also provides a target detection method, the method comprising: an target detection device acquiring an image to be detected, inputting the image to be detected into a student model, and identifying a target object from the image to be detected. The student model is a student model trained based on the detection model distillation method described above.

[0131] In one example, the target object is a license plate number, and the student model is used to recognize it. Users can input the acquired image to be detected into the trained student model. The student model detects vehicles in the image and automatically extracts the license plate region. It then performs character segmentation on the license plate region, recognizes the segmented characters, and obtains the license plate number. Based on the recognized license plate number and the location where the image was captured, license plates with the same number can be merged to obtain relevant information about the vehicle's movement curve, travel time, etc. Because the trained student model can quickly and efficiently complete the license plate number recognition task, it can be applied to scenarios such as parking fee management, traffic flow detection, highway toll management, vehicle speeding detection, vehicle route prediction, vehicle positioning, vehicle theft prevention, and red light violation detection, resulting in a better user experience.

[0132] In another example, the target object is a live person, and the student model is used to identify liveness. The user can input the acquired image to be detected into the trained student model. The student model detects and identifies liveness in the image, matching the identified liveness features with pre-stored personnel information to obtain the personnel information of the detected live person in the image. The trained student model can improve the speed of liveness detection and can be applied to scenarios such as entrance / exit personnel detection and querying specific personnel. When the trained student model is deployed in electronic products used for liveness detection, it can improve the practicality of those products.

[0133] In another example, the target object is a product, and the student model is used to detect whether the product has any appearance defects. During the product manufacturing process, due to factors such as raw materials, manufacturing processes, and the environment, products may develop various problems, a significant portion of which are appearance defects. Manually inspecting these defects is not only labor-intensive but also cannot guarantee accurate results. By inputting images of the product into a trained student model, appearance defects such as cracks, deformations, and missing parts can be quickly detected, improving product quality stability, increasing production efficiency, reducing labor costs, and minimizing the time spent on manual labor.

[0134] It should be understood that the above only exemplifies the scenarios in which the trained student model can be applied. The trained student model can also be applied to other target object detection scenarios, which will not be elaborated here.

[0135] In this way, by applying the trained student model to the object detection process, the efficiency and effectiveness of object detection can be improved, thus enhancing the user experience.

[0136] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0137] like Figure 9 As shown in the illustration, this application also provides a detection model distillation apparatus for the detection model distillation method shown in the above method embodiments. The detection model distillation apparatus 400 includes an acquisition module 401 and a processing module 402.

[0138] The acquisition module 401 is used to acquire a trained teacher model, sample images containing target objects, and ground truth values ​​of the target object recognition results corresponding to the sample images. The teacher model is used to output the recognition results of the target objects in the sample images. The processing module 402 is used to: input the sample images into the teacher model and the student model respectively; filter the recognition results output by the teacher model after processing the sample images to obtain positive teacher sample recognition results; for each positive teacher sample recognition result, determine the target anchor box that matches the positive teacher sample recognition result from multiple preset anchor boxes after the student model processes the sample images; perform logical output distillation on each matching combination to obtain the logical output distillation loss, which is used to indicate the degree of difference between the teacher model and the student model in the output layer; a matching combination includes a positive teacher sample recognition result and a target anchor box that matches the positive teacher sample recognition result; determine the detection task loss based on the ground truth value of the recognition result, which is used to indicate the degree of difference between the student model's recognition result of the target object and the ground truth value of the recognition result; and train the student model based on the logical output distillation loss and the detection task loss to obtain the trained student model.

[0139] In one possible implementation, the teacher positive sample identification result is a teacher positive sample box; the processing module 402 is specifically used to: for each anchor box, determine the intersection-union ratio (IUR) between the anchor box and the teacher positive sample box; and determine the anchor box indicated by the maximum value in the IUR as the target anchor box that matches the teacher positive sample box.

[0140] In one possible implementation, the processing module 402 is further configured to: correct the teacher positive sample recognition result based on the true value of the recognition result to obtain the corrected teacher positive sample recognition result; specifically, the processing module 402 is configured to: determine the area in the sample image that is marked by the modified teacher positive sample recognition result and not marked by the true value of the recognition result as the non-processing area, the non-processing area is used to instruct the student model not to perform target object recognition in the non-processing area; and obtain the detection task loss based on the true value of the recognition result and the non-processing area in the sample image.

[0141] In one possible implementation, the teacher positive sample recognition result is the teacher positive sample box and the teacher positive sample classification information corresponding to the teacher positive sample box, and the recognition result truth value is the recognition result truth value box and the truth value classification information corresponding to the recognition result truth value box; the processing module 402 is specifically used to: determine the intersection-union ratio between the recognition result truth value box and the teacher positive sample box; delete the teacher positive sample boxes in the teacher positive sample box whose intersection-union ratio is greater than a preset threshold and whose teacher positive sample classification information is different from the truth value classification information, and obtain the corrected teacher positive sample box.

[0142] In one possible implementation, a target anchor box corresponds to a student prediction box and student classification information. The student prediction box is obtained by the student model based on the target anchor box. The processing module 402 is specifically used to: for each matching combination, determine the localization loss between the student prediction box corresponding to the target anchor box and the corrected teacher positive sample box, and the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box; the localization loss indicates the degree of difference between the positions of the target objects identified by the teacher model and the student model, and the classification loss indicates the degree of difference between the classification information output by the teacher model and the student model; obtain the foreground and background confidence information of the teacher model, and the student model... The foreground and background confidence information of the models are used. The foreground and background of the teacher model represent the region output by the teacher model after processing the sample image, excluding the corrected teacher positive sample box. The foreground and background of the student model represent the region output by the student model after processing the sample image, excluding the student prediction box. Based on the foreground and background confidence information of the teacher model and the student model, the confidence loss between the foreground and background of the teacher model and the student model is determined. Based on the localization loss, the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box, and the confidence loss between the foreground and background of the teacher model and the student model, the logical output distillation loss is determined.

[0143] In one possible implementation, the feature layer of the student model includes a student backbone network and a student detection head, and the feature layer of the teacher model includes a teacher backbone network and a teacher detection head. The processing module 402 is further configured to: when the teacher detection head and student detection head have the same structure, perform feature distillation on the student backbone network using the teacher backbone network, and perform feature distillation on the student detection head using the teacher detection head, to obtain a feature distillation loss. The student backbone network is used to extract feature information from sample images and transmit it to the student detection head, which is used to detect target objects and their classification information from the feature information. The feature distillation loss is used to indicate the degree of difference between the teacher model and the student model in the feature layer. When the teacher detection head and student detection head have different structures, perform feature distillation on the student backbone network using the teacher backbone network, and perform feature distillation on the auxiliary detection head using the teacher detection head, to obtain a feature distillation loss. The auxiliary detection head has the same structure as the teacher detection head, and the student backbone network is also used to transmit feature information to the auxiliary detection head. Specifically, the processing module 402 is configured to train the student model based on the feature distillation loss, the logical output distillation loss, and the detection task loss, to obtain a trained student model.

[0144] like Figure 10As shown in the illustration, this application also provides a target detection device for the target detection method described in the above method embodiments. The target detection device 500 includes an acquisition module 501 and a processing module 502.

[0145] The acquisition module 501 is used to acquire the image to be detected; the processing module 502 is used to input the image to be detected into the student model and identify the target object from the image to be detected. The student model is trained based on any of the detection model distillation methods provided in the above embodiments.

[0146] It should be noted that, Figure 9 and Figure 10 The module division shown is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single processing module. These integrated modules can be implemented either in hardware or as software functional modules.

[0147] Another embodiment of this application also provides an electronic device, such as... Figure 11 As shown, the electronic device 600 includes a memory 601 and a processor 602; the memory 601 and the processor 602 are coupled; the memory 601 is used to store computer program code, which includes computer instructions. When the processor 602 executes the computer instructions, the electronic device 600 performs each step of the method flow shown in the above method embodiment.

[0148] In actual implementation, the acquisition module 401 and the processing module 402 can be composed of... Figure 11 The processor 602 shown calls the computer program code in memory 601 to implement this. The specific execution process can be found in the description of the detection model distillation method section above, and will not be repeated here.

[0149] In actual implementation, the acquisition module 501 and the processing module 502 can be composed of... Figure 11 The processor 602 shown calls the computer program code in memory 601 to implement this. The specific execution process can be found in the description of the target detection method section above, and will not be repeated here.

[0150] Another embodiment of this application provides a computer-readable storage medium storing computer instructions that, when executed on an electronic device, cause the electronic device to perform each step of the method flow shown in the above method embodiment.

[0151] In another embodiment of this application, a computer program product is also provided, which includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the various steps of the method flow shown in the above method embodiments.

[0152] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), etc.

[0153] The above description is merely a specific embodiment of this application. Any variations or substitutions conceived by those skilled in the art based on the specific embodiments provided in this application should be covered within the protection scope of this application.

Claims

1. A method for detecting model distillation, characterized in that, include: The teacher model is trained, a sample image containing the target object is obtained, and the ground truth value of the recognition result of the target object corresponding to the sample image is obtained. The teacher model is used to output the recognition result of the target object in the sample image. The sample images are input into the teacher model and the student model respectively. The recognition results output by the teacher model after processing the sample images are filtered to obtain the positive sample recognition results of the teacher. For each positive teacher sample recognition result, a target anchor box matching the positive teacher sample recognition result is determined from a set of preset anchor boxes after the student model processes the sample image; For each matching combination, logical output distillation is performed to obtain logical output distillation loss, which is used to indicate the degree of difference between the teacher model and the student model at the output layer; a matching combination includes a teacher positive sample recognition result and a target anchor box that matches the teacher positive sample recognition result. Based on the true value of the recognition result, the detection task loss is determined, which is used to indicate the degree of difference between the student model's recognition result of the target object and the true value of the recognition result; The student model is trained based on the logic output distillation loss and the detection task loss to obtain the trained student model.

2. The method according to claim 1, characterized in that, The teacher positive sample identification result is the teacher positive sample bounding box; The step of determining the target anchor box that matches the teacher's positive sample recognition result from a set of preset anchor boxes after processing the sample image from the student model includes: For each anchor frame, determine the intersection-union ratio (IoU) between the anchor frame and the teacher's positive sample frame; The anchor box indicated by the maximum value in the intersection-union ratio is determined as the target anchor box that matches the teacher's positive sample box.

3. The method according to claim 1, characterized in that, Before performing logical output distillation on each matching combination to obtain the logical output distillation loss, the method further includes: Based on the true value of the recognition result, the positive teacher sample recognition result is corrected to obtain the corrected positive teacher sample recognition result; The step of determining the detection task loss based on the true value of the recognition result includes: The region in the sample image where the corrected teacher positive sample recognition result is identified and the true value of the recognition result is not identified is defined as a non-processing region. The non-processing region is used to instruct the student model not to perform target object recognition in the non-processing region. Based on the true value of the recognition result and the unprocessed area in the sample image, the detection task loss is obtained.

4. The method according to claim 3, characterized in that, The teacher positive sample recognition result is the teacher positive sample box and the teacher positive sample classification information corresponding to the teacher positive sample box, and the recognition result truth value is the recognition result truth value box and the truth value classification information corresponding to the recognition result truth value box; Based on the true value of the recognition result, the teacher positive sample recognition result is corrected to obtain the corrected teacher positive sample recognition result, including: Determine the intersection-union ratio (IUU) between the ground truth bounding box of the recognition result and the teacher's positive sample bounding box; Delete the teacher positive sample frames that have an intersection-union ratio greater than a preset threshold and whose teacher positive sample classification information is different from the ground truth classification information, to obtain the corrected teacher positive sample frames.

5. The method according to claim 4, characterized in that, Each target anchor box corresponds to a student prediction box and a student classification information, wherein the student prediction box is obtained by the student model based on the target anchor box; The logical output distillation of each matching combination to obtain the logical output distillation loss includes: For each matching combination, the target anchor box is determined; the localization loss between the corresponding student predicted box and the corrected teacher positive sample box, and the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box; the localization loss is used to indicate the degree of difference between the positions of the target objects identified by the teacher model and the student model, and the classification loss is used to indicate the degree of difference between the classification information output by the teacher model and the student model. Obtain the foreground and background confidence information of the teacher model and the student model. The foreground and background of the teacher model are used to characterize the region output by the teacher model after processing the sample image, excluding the corrected teacher positive bounding box. The foreground and background of the student model are used to characterize the region output by the student model after processing the sample image, excluding the student prediction box. Based on the foreground confidence information of the teacher model and the foreground confidence information of the student model, the confidence loss between the foreground of the teacher model and the foreground of the student model is determined. The logical output distillation loss is determined based on the localization loss, the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box, and the confidence loss between the foreground and background of the teacher model and the foreground and background of the student model.

6. The method according to any one of claims 1-5, characterized in that, The feature layer of the student model includes a student backbone network and a student detection head, and the feature layer of the teacher model includes a teacher backbone network and a teacher detection head; the method further includes: When the teacher detection head and the student detection head have the same structure, feature distillation is performed on the student backbone network using the teacher backbone network, and feature distillation is performed on the student detection head using the teacher detection head to obtain a feature distillation loss. The student backbone network is used to extract feature information from the sample image and transmit it to the student detection head. The student detection head is used to detect the target object and its classification information from the feature information. The feature distillation loss is used to indicate the degree of difference between the teacher model and the student model at the feature layer. When the teacher detection head and the student detection head have different structures, the teacher backbone network is used to perform feature distillation on the student backbone network, and the teacher detection head is used to perform feature distillation on the auxiliary detection head to obtain the feature distillation loss. The auxiliary detection head has the same structure as the teacher detection head, and the student backbone network is also used to transmit the feature information to the auxiliary detection head. The process of training the student model based on the logical output distillation loss and the detection task loss to obtain the trained student model includes: The student model is trained based on the feature distillation loss, the logic output distillation loss, and the detection task loss to obtain the trained student model.

7. A target detection method, characterized in that, include: Acquire the image to be detected; The image to be detected is input into the student model, and the target object is identified from the image to be detected. The student model is trained based on the detection model distillation method of any one of claims 1-6.

8. A detection model distillation apparatus, characterized in that, include: The acquisition module is used to acquire a trained teacher model, a sample image containing a target object, and the ground truth value of the recognition result of the target object corresponding to the sample image. The teacher model is used to output the recognition result of the target object in the sample image. Processing module, used for: The sample images are input into the teacher model and the student model respectively. The recognition results output by the teacher model after processing the sample images are filtered to obtain the positive sample recognition results of the teacher. For each positive teacher sample recognition result, a target anchor box matching the positive teacher sample recognition result is determined from a set of preset anchor boxes after the student model processes the sample image; For each matching combination, logical output distillation is performed to obtain logical output distillation loss, which is used to indicate the degree of difference between the teacher model and the student model at the output layer; a matching combination includes a teacher positive sample recognition result and a target anchor box that matches the teacher positive sample recognition result. Based on the true value of the recognition result, the detection task loss is determined, which is used to indicate the degree of difference between the student model's recognition result of the target object and the true value of the recognition result; The student model is trained based on the logic output distillation loss and the detection task loss to obtain the trained student model.

9. The apparatus according to claim 8, characterized in that, The teacher positive sample recognition result is a teacher positive sample bounding box; the processing module is specifically used to: for each anchor box, determine the intersection-union ratio (IUR) between the anchor box and the teacher positive sample bounding box; and determine the anchor box indicated by the maximum value in the IUR as the target anchor box that matches the teacher positive sample bounding box; The processing module is further configured to: correct the teacher positive sample recognition result based on the ground truth value of the recognition result to obtain a corrected teacher positive sample recognition result; specifically, the processing module is configured to: determine the region in the sample image marked by the modified teacher positive sample recognition result and not marked by the ground truth value of the recognition result as a non-processing region, the non-processing region being used to instruct the student model not to perform target object recognition in the non-processing region; and obtain the detection task loss based on the ground truth value of the recognition result and the non-processing region in the sample image; The teacher positive sample recognition result is the teacher positive sample box and the teacher positive sample classification information corresponding to the teacher positive sample box. The recognition result truth value is the recognition result truth value box and the truth value classification information corresponding to the recognition result truth value box. The processing module is specifically used to: determine the intersection-union ratio (IUR) between the recognition result truth value box and the teacher positive sample box; delete teacher positive sample boxes in the teacher positive sample box where the IUR is greater than a preset threshold and the teacher positive sample classification information is different from the truth value classification information, to obtain the corrected teacher positive sample box. Each target anchor box corresponds to a student prediction box and a student classification information, wherein the student prediction box is obtained by the student model based on the target anchor box. The processing module is specifically used to: for each matching combination, determine the localization loss between the student prediction box corresponding to the target anchor box and the corrected teacher positive sample box, and the classification loss between the student classification information corresponding to the target anchor box and the teacher positive sample classification information corresponding to the corrected teacher positive sample box; the localization loss is used to indicate the degree of difference between the positions of the target objects identified by the teacher model and the student model, and the classification loss is used to indicate the degree of difference between the classification information output by the teacher model and the student model; obtain the foreground and background confidence information of the teacher model and the foreground and background confidence information of the student model. The foreground and background information of the teacher model are used to characterize the region output by the teacher model after processing the sample image, excluding the corrected teacher positive bounding box. The foreground and background information of the student model are used to characterize the region output by the student model after processing the sample image, excluding the student prediction box. Based on the foreground and background confidence information of the teacher model and the student model, the confidence loss between the foreground and background information of the teacher model and the student model is determined. Based on the localization loss, the classification loss between the student classification information corresponding to the target anchor box and the teacher positive bounding box corresponding to the corrected teacher positive bounding box, and the confidence loss between the foreground and background information of the teacher model and the student model, the logical output distillation loss is determined. The feature layer of the student model includes a student backbone network and a student detection head, and the feature layer of the teacher model includes a teacher backbone network and a teacher detection head. The processing module is further configured to: when the teacher detection head and the student detection head have the same structure, perform feature distillation on the student backbone network using the teacher backbone network, and perform feature distillation on the student detection head using the teacher detection head, to obtain a feature distillation loss. The student backbone network is used to extract feature information from the sample image and transmit it to the student detection head. The student detection head is used to detect the target object and its classification information from the feature information. The feature distillation loss... This is used to indicate the degree of difference between the teacher model and the student model at the feature layer; when the structures of the teacher detection head and the student detection head are different, feature distillation is performed on the student backbone network using the teacher backbone network, and feature distillation is performed on the auxiliary detection head using the teacher detection head to obtain the feature distillation loss. The auxiliary detection head has the same structure as the teacher detection head, and the student backbone network is also used to transmit the feature information to the auxiliary detection head; the processing module is specifically used to train the student model based on the feature distillation loss, the logic output distillation loss, and the detection task loss to obtain the trained student model.

10. A target detection device, characterized in that, include: The acquisition module is used to acquire the image to be detected; The processing module is used to input the image to be detected into the student model and identify the target object from the image to be detected. The student model is trained based on the detection model distillation method of any one of claims 1-6.

11. An electronic device, characterized in that, include: One or more processors; One or more memory units; Wherein, the one or more memories are used to store computer program code, the computer program code including computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the detection model distillation method according to any one of claims 1 to 6, or the target detection method according to claim 7.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed on a computer, cause the computer to perform the detection model distillation method according to any one of claims 1 to 6, or the target detection method according to claim 7.

Images