Model training method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a model training method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: performing model compression processing on a first target detection model to obtain a second target detection model; performing first target detection processing on the first image through a first target detection model to obtain a first target detection result, and performing second target detection processing on the first image through a second target detection model to obtain a second target detection result; obtaining a difference value between the second model loss and the first model loss, and obtaining a fusion parameter in positive correlation with the difference value; obtaining a comparison loss between the first target detection result and the second target detection result, and fusing the second model loss and the comparison loss based on the fusion parameter to obtain a fusion loss; and updating the second target detection model based on the fusion loss to obtain an updated second target detection model. According to the invention, the target detection model can be lightened, and the performance and detection precision of the model can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method, device, electronic device, storage medium and program product. Background Art

[0002] With the rapid development of artificial intelligence and deep learning technologies, AI-based visual algorithms have been widely adopted in various intelligent devices. Object detection, a core problem in computer vision, has been widely applied in various fields such as industrial production, transportation, and robotics, especially in household robots such as sweepers. During operation, sweepers must accurately detect obstacles, identify furniture, and distinguish floor materials in real time to ensure efficient and safe operation.

[0003] In actual deployment, robot vacuums place strict constraints on the computing resources and power consumption of object detection models. These models must be lightweight while maintaining high detection performance. Related technologies have reduced the number of parameters in object detection models to lower computational load and memory usage, but this also results in a certain degree of degradation in model detection performance. Summary of the Invention

[0004] The embodiments of the present application provide a model training method, device, electronic device, storage medium and program product, which can lightweight the target detection model and improve the performance and detection accuracy of the model.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] This embodiment of the present application provides a model training method, the method comprising:

[0007] Performing model compression processing on the first target detection model to obtain a second target detection model;

[0008] Performing a first target detection process on the first image using the first target detection model to obtain a first target detection result, and performing a second target detection process on the first image using the second target detection model to obtain a second target detection result;

[0009] Obtaining a first model loss corresponding to the first object detection result, and obtaining a second model loss corresponding to the second object detection result;

[0010] Obtaining a difference between the second model loss and the first model loss, and obtaining a fusion parameter positively correlated with the difference;

[0011] Obtaining a contrast loss between the first target detection result and the second target detection result, taking the fusion parameter as a weight of the contrast loss, and fusing the second model loss and the contrast loss based on the weight to obtain a fusion loss;

[0012] The second object detection model is updated based on the fusion loss to obtain an updated second object detection model.

[0013] The present invention provides a model training device, comprising:

[0014] a detection module configured to perform model compression processing on the first target detection model to obtain a second target detection model; perform first target detection processing on the first image using the first target detection model to obtain a first target detection result; and perform second target detection processing on the first image using the second target detection model to obtain a second target detection result;

[0015] A training module is used to obtain the difference between the second model loss and the first model loss, and obtain a fusion parameter positively correlated with the difference; obtain the contrast loss between the first target detection result and the second target detection result, use the fusion parameter as the weight of the contrast loss, and fuse the second model loss and the contrast loss based on the weight to obtain a fusion loss; update the second target detection model based on the fusion loss to obtain an updated second target detection model.

[0016] An embodiment of the present application provides an electronic device, comprising:

[0017] a memory for storing computer-executable instructions or computer programs;

[0018] The processor is used to implement the model training method provided in the embodiment of the present application when executing the computer-executable instructions or computer programs stored in the memory.

[0019] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the model training method provided in the embodiment of the present application when executed by a processor.

[0020] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions. When the computer program or computer-executable instructions are executed by a processor, the model training method provided in the embodiment of the present application is implemented.

[0021] The embodiments of the present application have the following beneficial effects:

[0022] By performing model compression on the first object detection model, the first object detection model is converted into a lightweight second object detection model, simplifying the object detection model structure and parameters. The first object detection model performs first object detection on the first image to obtain a first object detection result. The second object detection model then performs second object detection on the first image to obtain a second object detection result, which serves as the basis for comparing the differences between the first and second object detection models. The first model loss corresponding to the first object detection result and the second model loss corresponding to the second object detection result are obtained. The difference between the first and second model losses is calculated, and a fusion parameter positively correlated with the difference is obtained. Based on the fusion parameter, the contrastive loss between the first and second detection results is fused to determine the fusion loss. A contrastive loss constraint is introduced based on the fusion parameter, enabling the second object detection model to specifically learn the more optimal predictive response features from the first object detection model. The second object detection model is updated based on the fusion loss to obtain an updated second object detection model. When compression of the first object detection model results in a decrease in object detection capability and accuracy, the knowledge of the first object detection model is transferred to compensate the second object detection model, thereby enhancing the second object detection model's object detection capability in complex environments. While reducing redundant calculations, the optimization efficiency of the target detection model is improved, so that the compressed second target detection model meets the lightweight requirements while maintaining detection performance and accuracy similar to the first target detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of an application mode of the model training method provided in an embodiment of the present application;

[0024] Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0025] Figure 3 This is a first flow chart of the model training method provided in an embodiment of the present application;

[0026] Figure 4 This is a second flow chart of the model training method provided in an embodiment of the present application;

[0027] Figure 5 This is a schematic diagram of the model compression provided by the embodiment of the present application;

[0028] Figure 6 Schematic diagram of the target detection model provided in the embodiment of the present application;

[0029] Figure 7 This is a schematic diagram of the detection frame provided in an embodiment of the present application.

[0030] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of distinction between the advantages and disadvantages of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0033] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0034] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0035] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0036] In the embodiments of this application, the collection and processing of relevant data (for example, obstacle information) should be strictly in accordance with the requirements of relevant laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0037] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0038] 1) Object Detection: Object detection is a core task in computer vision. It aims to identify and locate specific objects in an image, outputting each object's category label and its precise location within the image (usually represented by a bounding box). Object detection simultaneously completes two subtasks: classification (determining the object category) and regression (predicting the bounding box coordinates). It is widely used in scenarios such as autonomous driving and robotic navigation.

[0039] 2) Knowledge Distillation: This is a model compression technology that transfers the knowledge from a large-scale, high-precision teacher model to a student model with fewer parameters, thereby compressing the model while maintaining the original detection performance as much as possible.

[0040] 3) Teacher Model: This model provides knowledge guidance during the knowledge distillation process. It is typically a high-precision model (such as a deep neural network) with a large number of parameters and a complex structure that has been fully trained. The teacher model does not participate in parameter updates during the distillation phase. Its role is to generate knowledge signals such as soft labels, feature maps, or attention maps through inference, providing richer supervision information for the student model.

[0041] 4) Student Model: This is the target model trained in knowledge distillation. It is typically a lightweight model (such as a pruned or quantized network) with a streamlined structure, few parameters, and high computational efficiency. The student model simultaneously learns the labels of the original training data and the knowledge of the teacher model by optimizing a loss function (such as a combination of task loss and distillation loss), achieving the ability to efficiently execute tasks with limited resources.

[0042] 5) Channel pruning: This is a structured model compression technique designed to reduce the number of model parameters and computational complexity while maintaining performance by removing redundant or unimportant channels in a neural network. Its core is to filter channels in the convolutional layer, removing those that contribute little to the model output. This results in a lighter model, faster inference, and reduced hardware resource requirements.

[0043] 6) Upper Bound Concept: This is a strategy to simplify the analysis or optimization process by finding the upper bound of the problem. In the embodiment of this application, the performance index of the reference model (teacher model) on a specific task is set as the performance upper bound. The knowledge transfer mechanism is triggered only when the actual performance of the student model exceeds the upper bound. By dynamically controlling the timing and intensity of knowledge transfer, interference with the student model's strengths is avoided, and targeted enhancement is performed on its weak links, thereby balancing the needs of model compression and accuracy maintenance.

[0044] Object detection is widely used in various fields such as industrial production, transportation, and robotics, especially in the field of household robots such as sweepers. During operation, sweepers need to detect obstacles in the environment in real time and accurately to ensure efficient and safe operation. In actual deployment, sweepers have strict restrictions on the computing resources and power consumption of the object detection model, requiring the object detection model to be lightweight while ensuring high detection performance. Related technologies reduce the computing load and memory usage by reducing the number of parameters in the object detection model. Through the technology of knowledge distillation, the knowledge in a large-scale, high-precision teacher model is transferred to a student model with fewer parameters to achieve lightweight object detection models. However, traditional distillation methods may have the problem of insufficient alignment of features or predicted responses between the teacher and student models. The reduction of parameters in the compressed model will affect the accuracy of model detection and reduce the performance of the compressed model.

[0045] The embodiments of the present application provide a model training method, a model training device, an electronic device, a computer-readable storage medium, and a computer program product, which can lightweight the target detection model and improve the performance and detection accuracy of the model.

[0046] The following describes exemplary applications of the electronic devices provided in the embodiments of the present application. The devices provided in the embodiments of the present application can be implemented as various types of terminals, such as laptops, tablet computers, desktop computers, set-top boxes, smartphones, smart speakers, smart watches, smart TVs, and in-vehicle terminals. They can also be implemented as servers. The following describes exemplary applications when the devices are implemented as terminals or servers.

[0047] See also Figure 1 , Figure 1 This is a schematic diagram of the application mode of the model training method provided in the embodiment of the present application, which is used to support a model training application. Figure 1 The server 200, network 300, terminal device 400 and database 500 are involved. The terminal device 400 is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0048] In some embodiments, the embodiments of the present application can be implemented collaboratively by a server and a terminal device. For example, the user may be a person skilled in the art. Server 200 is a server for model training, terminal device 400 is a terminal operated by the user, and image data to be detected is stored in database 500. Terminal device 400 sends a target detection model training request to server 200. Server 200 receives the target detection model training request and, using the model training method provided in the embodiments of the present application, performs model compression processing on the first target detection model to obtain a second target detection model. The first target detection model performs a first target detection process on the image to be detected to obtain a first target detection result. The second target detection model performs a second target detection process on the image to be detected to obtain a second target detection result. A first model loss corresponding to the first target detection result and a second model loss corresponding to the second target detection result are obtained. A fusion parameter positively correlated with the difference between the two model losses is obtained based on the difference between the two model losses. The contrast loss and the second model loss between the detection results are fused based on the fusion parameter to obtain a fusion loss. The second target detection model is trained based on the fusion loss to obtain an updated second target detection model. The trained target detection model is then sent to terminal device 400.

[0049] In some embodiments, the model training method of the embodiments of the present application can also be applied in the following application scenarios:

[0050] 1. Obstacle detection scenario of a sweeping robot. For example, a terminal device sends a target detection model training request in the sweeping robot to a server. The server compresses the target detection model in the sweeping robot through the model training method provided in the embodiment of the present application to obtain a compressed target detection model. The target detection model is used to perform target detection processing on the image to be detected to obtain a target detection result, and a first model loss and a second model loss corresponding to the target detection result are obtained. A fusion parameter positively correlated with the difference between the two model losses is obtained according to the difference between the two model losses. The contrast loss and the second model loss between the detection results are fused based on the fusion parameter to obtain a fusion loss and the compressed target detection model is trained based on the fusion loss to obtain an updated compressed target detection model, thereby improving the detection accuracy of the sweeping robot for obstacles and realizing the lightweighting of the target detection model in the sweeping robot.

[0051] 2. Obstacle detection scenario of unmanned vehicles. For example, the terminal device sends a target detection model training request in the unmanned vehicle to the server. The server compresses the target detection model in the unmanned vehicle through the model training method provided in the embodiment of the present application to obtain a compressed target detection model. The target detection model is used to perform target detection processing on the image to be detected to obtain a target detection result, and a first model loss and a second model loss corresponding to the target detection result are obtained. A fusion parameter positively correlated with the difference between the two model losses is obtained based on the difference. The contrast loss and the second model loss between the detection results are fused based on the fusion parameter to obtain a fusion loss. The compressed target detection model is updated based on the fusion loss to obtain an updated compressed target detection model, thereby improving the detection accuracy of the unmanned vehicle for obstacles and realizing the lightweighting of the target detection model in the unmanned vehicle.

[0052] See also Figure 2 , Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application, Figure 2 The server 200 shown includes: at least one processor 410, a memory 450 and at least one network interface 420. The various components in the server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 2 Various buses are labeled as bus system 440 .

[0053] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0054] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0055] The memory 450 includes volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0056] In some embodiments, the memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplified below.

[0057] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0058] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420 . Exemplary network interfaces 420 include Bluetooth, Wireless LAN (WiFi), and Universal Serial Bus (USB).

[0059] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 2 Model training device 455 stored in memory 450 is shown. This device can be software in the form of a program or plug-in, and includes the following software modules: a detection module 4551 and a training module 4552. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0060] In some embodiments, the terminal or server can implement the model training method provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in an operating system; it can be a local (Native) application (APPlication, APP); it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0061] The model training method provided in the embodiments of the present application will be explained in combination with the exemplary application and implementation of the electronic device provided in the embodiments of the present application.

[0062] Below, the model training method provided by the embodiment of the present application is described. As mentioned above, the electronic device that implements the model training method of the embodiment of the present application can be a terminal, a server, or a combination of the two. Therefore, the execution entity of each step will not be repeated below.

[0063] See also Figure 3 , Figure 3 This is a first flow chart of the model training method provided in the embodiment of the present application, which will be combined with Figure 3 The steps shown are explained, Figure 3 The executive body is Figure 1 Server 200 in.

[0064] In step 301, model compression processing is performed on the first target detection model to obtain a second target detection model.

[0065] As an example, the first target detection model is obtained by training based on a known target detection model. The known target detection model is trained using a large-scale obstacle detection dataset to obtain the first target detection model. The first target detection models obtained by training with different types of datasets have different types of target recognition. For example: if the obstacle dataset is a dataset of shoe information, the trained first target detection model can recognize shoes. In an embodiment of the present application, the known target detection model can be a YOLOv6-n model, and this application does not impose any restrictions on this. The YOLOv6-n model is a lightweight network architecture variant in the YOLOv6 target detection framework. The network structure includes three parts: a backbone network, a feature fusion network, and a detection head.

[0066] In some embodiments, step 301 can be implemented by the following method: performing the following processing for each channel in the first target detection model: obtaining the weight parameters of the channel, and performing norm calculation processing on the weight parameters to obtain the importance of the channel; based on the importance of each channel of the first target detection model, pruning the first target detection model to obtain the second target detection model.

[0067] As an example, a first object detection model contains multiple channels. For each channel, each output from the convolutional layer is extracted and integrated into a corresponding weight parameter matrix. The norm of each output channel's weight parameter is calculated using the Euclidean norm, which is the square root of the sum of the squares of the vector elements. The calculated result is used as a quantitative indicator of the importance of the output channel in the feature extraction process, namely the channel importance. A higher importance value indicates a greater contribution to feature expression. Based on the importance of each channel in the first object detection model, the first object detection model is pruned. Channel pruning is a structured model compression technique that reduces the number of model parameters and computational complexity by removing redundant or unimportant channels in a neural network. After channel pruning is completed, the weight parameter matrix of the current convolutional layer is dimensionality reduced, and the number of input channels in subsequent network layers is adjusted to match the pruned output channels. Finally, the above channel screening and parameter adjustment operations are iteratively performed layer by layer to obtain the second object detection model.

[0068] Through the embodiments of the present application, the importance of each channel in the first target detection model is calculated, the importance of the channel in feature extraction is quantitatively evaluated, high-contribution channels are screened and retained, structured pruning is implemented based on the channel importance, redundant channels are removed to reduce the number of model parameters and computational complexity, and the synergistic performance of the backbone network, feature fusion network and detection head module is maintained in the compressed second target detection model, so that the lightweight model can still extract effective features through key channels and complete multi-scale obstacle detection while reducing memory usage and computational load, while effectively controlling the loss of model accuracy, providing reliable technical support for efficient obstacle positioning and classification in complex scenarios.

[0069] In some embodiments, based on the importance of each channel of the first target detection model, the first target detection model is pruned to obtain a second target detection model. This can be achieved by the following method: based on the importance of each channel, the channels are sorted in descending order to obtain a channel sorting result; from the channel sorting result, channels that meet a preset pruning ratio are filtered out in order from front to back; the channels filtered out from the first target detection model are removed to obtain a second target detection model.

[0070] As an example, the importance of each channel in the first target detection model is sorted in descending order to generate a channel sorting result. The sorting result reflects the contribution priority of different channels in the feature extraction task, and the retained channels are screened out from the channel sorting result in order from front to back according to the preset pruning ratio. The low-importance channels that are not screened and retained in the first target detection model are removed, and the dimension of the weight parameter matrix of the corresponding convolutional layer is adjusted synchronously to ensure that the number of output channels is consistent with the number of retained channels after screening. The number of input channels of the subsequent network layers is updated layer by layer to match the number of output channels after pruning of the current layer, to avoid feature transfer anomalies caused by channel number mismatch. By iteratively performing the above adjustment operation, a second target detection model with a simplified parameter amount is generated.

[0071] For example, when the pruning ratio is set to 30%, the original layer is: convolutional layer A, the number of output channels is 100, the weight shape is [100, 64, 3, 3], 70 channels are retained, the importance of the 100 channels is calculated and sorted, the first 70 high-scoring channels are retained, and the last 30 are removed. The weight of layer A is adjusted to [70, 64, 3, 3], then the number of input channels of the next layer B changes from 100 to 70, and its weight shape is adjusted from [128, 100, 3, 3] to [128, 70, 3, 3] (deleting the corresponding 30 pruned input channels).

[0072] Through the embodiments of the present application, the importance of each channel in the first target detection model is sorted in descending order, the contribution priority of the channel in the feature extraction task is determined, and the high-importance channels are retained and the low-importance channels are removed through a preset pruning ratio. Through the importance-oriented channel pruning strategy, while reducing the model memory usage and computing resource consumption, the key channels retain the ability to express obstacle features, and the conflict between model lightweighting and detection performance is balanced, so that the second target detection model retains the ability of key channels to represent obstacle features while reducing memory usage and computing resource consumption.

[0073] In step 302, a first target detection process is performed on the first image using a first target detection model to obtain a first target detection result, and a second target detection process is performed on the first image using a second target detection model to obtain a second target detection result.

[0074] As an example, a first target detection model is used to perform first target detection processing on a first image to obtain a first target detection result, which includes a detection box, a category, and a confidence level. The detection box (Bounding Box) is the prediction result of the object position in the first target detection model, which represents the position and size of the object in the image in the form of a rectangular box. The confidence level (Confidence) is the degree of confidence of the first target detection model that the target object exists in the detection box and that the category is correct. It is usually expressed as a numerical value between 0 and 1, and the closer the value is to 1, the higher the confidence level. The second target detection model is used to perform detection processing on the image to be detected to obtain a second target detection result, including a detection box and the corresponding category and confidence information. The principles of detection processing by the first target detection model and the second target detection model are the same.

[0075] In some embodiments, see Figure 6 , Figure 6 It is a structural diagram of the target detection model provided in an embodiment of the present application; the image to be detected 601 is input into the target detection model 602, and the image to be detected 601 is usually adjusted to a fixed size, such as 640×640 pixels, to meet the input requirements of the target detection model 602. The structure of the target detection model 602 includes three parts: a backbone network 6021, a feature fusion network 6022 and a detection head 6023. The structures of the first target detection model and the second target detection model are the same as the structure of the target detection model 602. The backbone network 6021 extracts multi-level features from the input image to be detected 601 (i.e., the first image) and constructs a basic feature expression. The feature fusion network 6022 fuses feature maps of different levels to enhance the multi-scale target detection capability. The detection head 6023 generates the final detection box, category and confidence, and outputs them as the detection result 603 (the first target detection result and the second target detection result).

[0076] In some embodiments, see Figure 7 , Figure 7 Schematic diagram of the detection frame provided in the embodiment of the present application; Figure 7 Two detection boxes and the corresponding target detection results are shown in the figure. The detection result 7011 indicates that the category of the detection result of the image displayed in the detection box 701 is shoes, and the confidence result is 94.4%, indicating that there is a 94.4% probability that the image displayed in the detection box 701 is shoes. The detection result 7021 indicates that the category of the detection result of the image displayed in the detection box 702 is shoes, and the confidence result is 89.1%, indicating that there is an 89.1% probability that the image displayed in the detection box 702 is shoes.

[0077] Through the embodiment of the present application, target detection processing is performed on the first image through the first target detection model and the second target detection model respectively, and independent target detection results can be obtained. The target detection results all contain detection boxes, categories, and confidence information. The first target detection model and the second target detection model have the same structure. During the processing, the input first image will be adjusted to a fixed size to meet the model input requirements. The two models can generate detection results with independent confidence evaluation based on their internal feature extraction, fusion, and detection mechanisms. By comparing the detection results of the two models, the accuracy of the detection results can be verified, providing a basis for subsequent knowledge distillation.

[0078] In step 303, a first model loss corresponding to the first target detection result is obtained, and a second model loss corresponding to the second target detection result is obtained.

[0079] In some embodiments, obtaining the first model loss corresponding to the first target detection result can be achieved by the following method: extracting the first probability distribution corresponding to each candidate category under the category dimension from the first target detection result, and extracting the first predicted position corresponding to the detection box under the position dimension from the first target detection result; performing difference calculation processing on the first probability distribution corresponding to each candidate category and the true probability distribution corresponding to each candidate category to obtain the loss corresponding to the category dimension; performing difference calculation processing on the first predicted position corresponding to the detection box and the first true position corresponding to the detection box to obtain the loss corresponding to the position dimension; performing fusion processing on the loss corresponding to the position dimension and the loss corresponding to the category dimension to obtain the first model loss corresponding to the first target detection result.

[0080] As an example, the first target detection result includes detection results in the category dimension and detection results in the position dimension. The first probability distribution corresponding to each candidate category in the category dimension is extracted from the first target detection result. The first probability distribution reflects the probability distribution of the target category predicted by the first target detection model for each candidate detection frame. The first predicted position corresponding to the detection frame in the position dimension is extracted from the first target detection result. The first predicted position is the model's prediction result of the position of the detection frame in the image. The first probability distribution corresponding to each candidate category is subjected to a difference calculation process with the corresponding true probability distribution. The true probability distribution is determined based on the labeled data and represents the true situation of the target category in each candidate detection frame. By calculating the difference between the first probability distribution and the true probability distribution, the loss corresponding to the category dimension is obtained. The first predicted position corresponding to the detection frame is subjected to a difference calculation process with the corresponding first true position. The first true position is the true position of the detection frame in the image determined based on the labeled data. By calculating the difference between the first predicted position and the first true position, the loss corresponding to the position dimension is obtained.

[0081] The loss corresponding to the position dimension and the loss corresponding to the category dimension are fused to obtain the first model loss corresponding to the first target detection result, which can be achieved by formula (1), as described in detail below.

[0082] L T =L box +L cls (1)

[0083] Among them, L T is the first model loss, L box is the loss corresponding to the position dimension between the first predicted position of the detection box and the true annotation box, L cls Is the loss corresponding to the category dimension between the first probability distribution and the true category. The loss corresponding to the position dimension (L box ) can be calculated by the intersection over union loss. The intersection over union loss (IoU Loss) directly uses the intersection over union (IoU) of the detection box and the true annotation box as the optimization target, and maximizes the overlapping area between the detection box and the true annotation box by minimizing 1-IoU. The intersection over union value range is [0, 1]. The smaller the loss value, the closer the position and size of the detection box and the true annotation box are. The loss corresponding to the category dimension (L cls ) can be determined by calculating the cross entropy loss between the first probability distribution and the true category, which is not limited in this application.

[0084] Through the embodiments of the present application, the category dimension loss is calculated by the difference in probability distribution, reflecting the uncertainty of the first target detection model in predicting the target category. The position dimension loss is calculated by the intersection-over-union loss of the detection box and the true annotation box, reflecting the accuracy of the detection box positioning. The loss corresponding to the position dimension and the loss corresponding to the category dimension are added together, and the position accuracy of the detection box and the accuracy of the category prediction are comprehensively considered to comprehensively reflect the overall performance of the first target detection model in the target detection task.

[0085] In some embodiments, obtaining the second model loss corresponding to the second target detection result can be achieved by the following method: extracting the second probability distribution corresponding to each candidate category under the category dimension from the second target detection result, and extracting the second predicted position corresponding to the detection box under the position dimension from the second target detection result; performing difference calculation processing on the second probability distribution corresponding to each candidate category and the true probability distribution corresponding to each candidate category to obtain the loss corresponding to the category dimension; performing difference calculation processing on the second predicted position corresponding to the detection box and the second true position corresponding to the detection box to obtain the loss corresponding to the position dimension; performing fusion processing on the loss corresponding to the position dimension and the loss corresponding to the category dimension to obtain the second model loss corresponding to the second target detection result.

[0086] As an example, the second target detection result includes the detection result of the category dimension and the detection result of the position dimension. The loss corresponding to the position dimension and the loss corresponding to the category dimension are fused to obtain the second model loss corresponding to the second target detection result. This can be achieved by formula (2), which is described in detail below.

[0087] L S =L box +L cls (2)

[0088] Among them, L S is the second model loss, L box is the loss corresponding to the position dimension between the second predicted position of the detection box and the true annotation box, L cls It is the loss corresponding to the category dimension between the second probability distribution and the true category. The calculation principle of the second model loss is the same as that of the first model loss, so it will not be repeated here.

[0089] Through the embodiments of the present application, the performance of the second target detection model in the detection task is quantified by determining the second model loss. The calculation of the category dimension loss can evaluate the model's ability to recognize the target category. The smaller the difference in probability distribution, the more accurate the category prediction. The calculation of the position dimension loss can evaluate the positioning accuracy of the detection frame. The higher the intersection-over-union ratio, the higher the degree of overlap between the detection frame and the true annotation frame. The two are integrated into a comprehensive second target detection model loss to comprehensively reflect the overall performance of the second target detection model in the target detection task, providing a clear direction for the optimization of the second target detection model.

[0090] In step 304, the difference between the second model loss and the first model loss is obtained, and a fusion parameter positively correlated with the difference is obtained.

[0091] As an example, the difference between the second model loss and the first model loss is calculated, and the fusion parameter λ is dynamically adjusted according to the size of the difference. The first model loss is used as the upper bound. When the difference is a positive value, the first model loss is less than the second model loss, that is, the second model loss (L S ) is not less than the upper bound, and a fusion parameter positively correlated with the difference is obtained. For example, the threshold for the difference between the first model loss and the second model loss is set to 0.2. When the difference is less than the difference threshold, the fusion parameter is linearly increased. The fusion parameter is positively correlated with the difference. When the difference is greater than the difference threshold, the fusion parameter λ = 1. When the difference is a negative value, the first model loss is not less than the second model loss, and the fusion parameter λ = 0.

[0092] In step 305, the contrast loss between the first target detection result and the second target detection result is obtained, and the fusion parameter is used as the weight of the contrast loss. The second model loss and the contrast loss are fused based on the weight to obtain the fusion loss.

[0093] In some embodiments, obtaining the contrast loss between the first target detection result and the second target detection result can be achieved by the following method: extracting the first probability distribution corresponding to each candidate category under the category dimension from the first target detection result, and extracting the second probability distribution corresponding to each candidate category under the category dimension from the second target detection result; performing the following processing for each candidate category under the category dimension: performing divergence calculation processing on the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category; performing fusion processing on the contrast losses corresponding to multiple candidate categories to obtain the contrast loss between the first target detection result and the second target detection result.

[0094] As an example, when the loss of the first model is less than the loss of the second model, the loss of the first model is used as the upper bound to obtain the contrast loss between the first target detection result and the second target detection result. The upper bound idea is a strategy to simplify the analysis or optimization process by finding the upper limit constraint of the problem. By determining the maximum possible value (upper bound) of a parameter or variable, the feasible domain of the problem is narrowed to avoid infinite exploration. When the loss of the second model exceeds the upper bound, the divergence calculation is performed on the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category. Each detection box contains multiple candidate categories, and the contrast losses corresponding to the multiple candidate categories are fused to obtain the contrast loss between the first target detection result and the second target detection result.

[0095] Through the embodiments of the present application, when the loss of the second model is greater than the loss of the first model, the divergence calculation is performed on the first probability distribution and the second probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to each candidate category, and the contrast losses of multiple categories are fused to obtain the final contrast loss. The divergence calculation can measure the degree of difference between the two probability distributions and reflect the similarities and differences in category prediction between the two target detection models. By fusing the contrast losses corresponding to multiple candidate categories, a comprehensive contrast loss value can be obtained, which fully reflects the difference between the two target detection models in overall category prediction.

[0096] In some embodiments, a divergence calculation is performed on the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category. This can be achieved by the following method: determining the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category to calculate the ratio to obtain a first ratio; performing logarithmic calculation on the first ratio to obtain a first logarithmic result; multiplying the first logarithmic result by the first probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category.

[0097] As an example, the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category are determined to perform ratio processing to obtain a first ratio, and the first ratio is logarithmically calculated to obtain a first logarithmic result. The first logarithmic result reflects the degree of prediction deviation between the first probability distribution and the second probability distribution on the category. The first logarithmic result is multiplied by the first probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category. This can be achieved by formula (3), which is explained in detail below.

[0098]

[0099] Among them, P T is the first probability distribution, which is the predicted probability value of the target object in the current detection frame belonging to the i-th category. The probability distribution is normalized, P S is the second probability distribution, i is the i-th category, and represents the index of all possible categories in the target detection task. Calculate the natural logarithm of the probability distribution ratio of the first detection model and the second detection model in category i (the first logarithm result), which reflects the degree of deviation between the two predictions in this category. KL(P T ||P S ) is the logarithmically weighted sum of the ratios of the first probability distribution and the second probability distribution of all categories, which quantifies the information loss of the output distribution of the second target prediction model relative to the distribution of the first target prediction model, and obtains the contrast loss corresponding to the candidate category.

[0100] Through the embodiments of the present application, the contrast loss determined by the divergence calculation intuitively reflects the information loss introduced when one distribution is approximated by another distribution, guides the target detection model selection or fusion strategy, improves the overall performance of the detection system, can effectively evaluate the differences in category prediction between two target detection models, and provide a scientific basis for model optimization and selection.

[0101] In some embodiments, the fusion parameter is used as the weight of the contrast loss, and the second model loss and the contrast loss are fused based on the weight to obtain the fusion loss. This can be achieved by the following method: multiplying the fusion parameter with the contrast loss to obtain the multiplication result, and fusing the multiplication result with the second model loss to obtain the fusion loss.

[0102] As an example, the fusion parameter is multiplied by the contrast loss to obtain the multiplication result, the influence of the contrast loss on the fusion loss is adjusted by the fusion parameter, and the multiplication result is fused with the second model loss to obtain the fusion loss. This can be achieved by formula (4), which is described in detail below.

[0103] l total =l S +λ·KL(P T ||P S ) (4)

[0104] Among them, L total is the fusion loss function for training the second target detection model, l S is the second model loss, λ is the fusion parameter, KL(P T ||P S ) is the contrast loss between the first target detection model and the second target detection model, λ·KL(P T ||P S ) is the product of the fusion parameter and the contrastive loss. When the fusion parameter λ = 0, the fusion loss is equal to the second model loss. That is, the second object detection model is updated based solely on the second model loss to obtain an updated second object detection model. When the fusion parameter λ = 1 or other values ​​positively correlated with the difference, the second model loss is added to the second model loss, and the impact of the contrastive loss on the fusion parameter is considered. The second object detection model is trained based on the second model loss and the contrastive loss.

[0105] Through the embodiments of the present application, the weight of the contrast loss in the fusion loss is controlled by fusion parameters, and adaptive adjustment of the fusion loss to the training target is achieved. Through the parameterized fusion mechanism, the coordinated optimization of task performance and knowledge transfer is achieved, providing an efficient and flexible technical path for distillation training.

[0106] In step 306, the second object detection model is updated based on the fusion loss to obtain an updated second object detection model.

[0107] As an example, the second target detection model is updated based on the fusion loss, and backpropagation is performed on the second target detection model according to the fusion loss to calculate the gradient of the trainable parameters of the second target detection model relative to the fusion loss. The gradient reflects the direction and magnitude of the parameter adjustment. The optimizer is combined with the calculated gradient to update the parameters of the second target detection model according to the preset learning rate. The model parameters are continuously updated on multiple training batches until the fusion loss converges or the preset training termination condition is reached. When the training is completed, the second target detection model with updated parameters is saved to obtain the updated second target detection model. By fusion loss, it is ensured that only the loss of the second model (L S ) exceeds the loss of the first model, knowledge distillation constraints are applied to it.

[0108] The model training method provided by the embodiments of the present application has the following beneficial effects:

[0109] The importance of each channel in the first object detection model is calculated to quantitatively assess its contribution to feature extraction. Low-importance redundant channels are preferentially removed, significantly reducing the number of parameters and computational complexity of the second object detection model. This approach reduces memory usage and inference latency while preserving the key channels' ability to represent obstacle features, achieving a balance between lightweight object detection models and detection accuracy. The compressed second object detection model retains the key channels' ability to represent obstacle features while reducing memory usage and computational resource consumption. Through the collaborative training of the first object detection model (teacher model) and the second object detection model (student model), a contrastive loss based on divergence calculation is introduced. This contrastive loss intuitively reflects the information loss introduced when approximating one distribution with another, guiding object detection model selection or fusion strategies, improving the overall performance of the detection system, and effectively evaluating the differences in class prediction between the two object detection models. The class probability distribution of the second object detection model is forced to align with the output distribution of the first object detection model, leveraging the high-precision detection capabilities of the first object detection model trained on large datasets to guide the optimization of the second object detection model. When the loss of the second model exceeds the loss of the first model, the difference between the models is calculated, and the fusion parameters are determined based on the difference. The fusion loss used to train the second target detection model is determined based on the fusion parameters. Only when the loss of the second model exceeds the loss of the first model, the contrastive loss is introduced to constrain the knowledge distillation. When the loss of the second model does not exceed the loss of the first model, only the loss of the second model itself is used for updating. The weight of the contrastive loss in the fusion loss is controlled by the fusion parameters, and the adaptive adjustment of the fusion loss to the training target is achieved. Through the parameterized fusion mechanism, the coordinated optimization of task performance and knowledge transfer is achieved. The fusion parameters ensure that the knowledge distillation constraint is only applied when the loss of the second model exceeds the loss of the first model, avoiding excessive punishment. This helps to maintain the flexibility of the second target detection model, so that the updated second target detection model can achieve lightweight model parameters while maintaining target detection accuracy.

[0110] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0111] With the rapid development of artificial intelligence and deep learning technologies, AI-based visual algorithms have been widely adopted in various smart devices, particularly in household robots such as sweepers. During operation, sweepers must accurately detect obstacles in the environment, identify furniture, and distinguish floor materials in real time to ensure efficient and safe operation. In actual deployments, sweepers impose strict constraints on the computing resources and power consumption of object detection models, requiring them to be lightweight while maintaining high detection performance. Related technologies use pruning to reduce the number of parameters in object detection models, lowering the computational load and memory usage. Using knowledge distillation, they transfer knowledge from a large-scale, high-precision teacher model to a student model with fewer parameters, achieving lightweight object detection models. However, compressing the model and reducing the number of parameters can affect the accuracy of model detection and reduce the performance of the compressed model.

[0112] The embodiment of the present application trains a high-performance target detection model using a large-scale dataset, detects obstacles based on the pre-trained target detection model, outputs the obstacle detection results, including the prediction box, the corresponding category, and the execution degree information, and prunes the target detection model as a teacher model to obtain a lightweight student model. The difference in the output distribution of the teacher model and the student model is calculated to determine the probability distribution loss between the teacher model and the student model. The regression loss of the teacher model and the student model is determined separately, and the regression loss of the teacher model is used as the upper bound. Only when the error of the student model exceeds the upper bound, the student model is subjected to knowledge distillation, and the regression loss and probability distribution loss of the student model itself are used as the total loss. The student model is trained to obtain a trained lightweight target detection model, thereby achieving a lightweight model while improving the detection performance of the model.

[0113] The following is a description with reference to the accompanying drawings. Figure 4 , Figure 4 This is a second flow chart of the model training method provided in the embodiment of the present application. The execution subject can be a terminal device, a server, or a combination of the two. The embodiment of the present application takes the server as the execution subject as an example. Figure 4 The steps shown are explained in detail.

[0114] In step 401, a first detection model is trained based on an obstacle dataset to obtain a trained first detection model.

[0115] As an example, a large-scale obstacle detection dataset is used to train the first detection model. The first detection model obtained by training different types of datasets has different types of target recognition. For example, if the obstacle dataset is a dataset of shoe information, the trained detection model can recognize shoes. The first detection model is a target detection model. In an embodiment of the present application, the first detection model can be a YOLOv6-n model, which is not limited in this application. The YOLOv6-n model is a lightweight network architecture variant in the YOLOv6 target detection framework. The YOLOv6-n model is based on the YOLOv6 infrastructure and reduces the number of parameters by compressing the network depth (number of layers) and width (number of channels). It significantly reduces the computational complexity and memory usage while ensuring the accuracy of multi-task detection. The network structure includes three parts: a backbone network, a feature fusion network, and a detection head. The backbone network extracts multi-scale features of the image by stacking lightweight convolutional units. The feature fusion network uses cross-level connections to enhance the ability to detect small targets. The detection head outputs the coordinates, category probability, and confidence information of the prediction box through classification and regression branches.

[0116] In step 402, the first detection model is compressed according to a preconfigured ratio to obtain a second detection model.

[0117] As an example, the first detection model is compressed according to a preconfigured ratio, and the first detection model is compressed based on the idea of ​​knowledge distillation to obtain a second detection model after the compression model. Knowledge distillation is a model compression technology that transfers the knowledge in a large-scale, high-precision teacher model to a student model with fewer parameters, thereby compressing the model while maintaining the original detection performance as much as possible. In the embodiment of the present application, the first detection model is the teacher model. The teacher model is a model that provides knowledge guidance in the knowledge distillation process. It is usually a high-precision model with a large number of parameters, a complex structure and sufficient training. The teacher model does not participate in parameter updates in the distillation stage. Its role is to generate knowledge signals such as soft labels, feature maps or attention maps through reasoning to provide richer supervision information for the student model. The second detection model is the student model. The student model is the target model trained in the knowledge distillation. It is usually a lightweight model with a streamlined structure, a small number of parameters and high computational efficiency (such as a pruned or quantized network). The student model learns the labels of the original training data and the knowledge of the teacher model at the same time by optimizing the loss function (such as combining task loss and distillation loss), thereby achieving the ability to efficiently perform tasks under limited resources.

[0118] In some embodiments, see Figure 5 , Figure 5: This is a schematic diagram of model compression provided in an embodiment of the present application; the target detection model is used as the original teacher model 501, and is trained using an obstacle dataset to obtain a trained teacher model 5011. The trained teacher model 5011 can detect the corresponding obstacle category in the obstacle dataset. The trained teacher model 5011 is compressed to obtain a compressed student model 502. The compressed student model 502 is a model obtained by reducing the model structure and parameters on the basis of the trained teacher model 5011. The knowledge distillation framework 503 is applied to the trained teacher model 5011 and the compressed student model 502 for training to obtain a trained target model 504. The target model 504 is a target detection model that has been lightweight processed while maintaining the performance and detection accuracy of the model.

[0119] As an example, the compression processing of the first detection model can be achieved through the channel pruning method. Channel pruning is a structured model compression technology that reduces the number of model parameters and computational complexity by removing redundant or unimportant channels in the neural network. The channels of the first detection model are pruned using the Euclidean norm. The Euclidean norm of the weight of each output channel in the first detection model is calculated as the importance score of each output channel. The high-scoring channels are retained according to the preset pruning ratio, and the low-scoring channels are pruned to obtain the second detection model.

[0120] The pruning ratio determines the number of channels removed from each convolutional layer in the first detection model. A higher pruning ratio reduces model parameters and computational complexity. An importance score is calculated for each output channel of the convolutional layer. All output channels are sorted from high to low according to their importance scores. High-scoring channels are retained based on the pruning ratio, while the remaining low-scoring channels are removed. For example, consider the original layer: convolutional layer A, with 100 output channels and a weight shape of [100, 64, 3, 3]. With a pruning ratio of p = 30%, 70 channels are retained. The L2 norm of the 100 channels is calculated and sorted, the top 70 high-scoring channels are retained, and the bottom 30 are removed. The weights of layer A are adjusted to [70, 64, 3, 3]. The number of input channels of the next layer, B, changes from 100 to 70, and its weight shape changes from [128, 100, 3, 3] to [128, 70, 3, 3] (deleting the corresponding 30 pruned input channels).

[0121] Continue to see Figure 4 In step 403, the image to be detected is detected and processed by the first detection model to obtain a first detection result, and the image to be detected is detected and processed by the second detection model to obtain a second detection result.

[0122] As an example, the trained first detection model is used to perform prediction processing on the image to be detected, and obstacles are detected to obtain a first detection result, including a detection box (Bounding Box), a category (Cla ss) and a confidence (Confidence). The detection box is the prediction result of the object position in the first detection model, which represents the position and size of the object in the image in the form of a rectangular box. The confidence is the degree of confidence of the first detection model that the target object exists in the detection box and the category is correct. It is usually expressed as a numerical value between 0 and 1. The closer the value is to 1, the higher the degree of confidence. The second detection model is used to perform detection processing on the image to be detected, and a second detection result is obtained, including a detection box and the corresponding category and confidence information. The principles of detection processing by the first detection model and the second detection model are the same.

[0123] In some embodiments, see Figure 6 , Figure 6 It is a structural diagram of the target detection model provided in an embodiment of the present application; the image to be detected 601 is input into the target detection model 602, and the image is usually adjusted to a fixed size, such as 640×640 pixels, to meet the input requirements of the target detection model 602. The structure of the target detection model 602 includes three parts: a backbone network 6021, a feature fusion network 6022 and a detection head 6023. The structures of the first detection model and the second detection model are the same as those of the target detection model. The backbone network 6021 extracts multi-level features from the input image to be detected 601 and constructs a basic feature expression. The feature fusion network 6022 fuses feature maps of different levels to enhance the multi-scale target detection capability. The detection head 6023 generates the final detection box, category and confidence, and outputs them as the detection result 603.

[0124] In some embodiments, see Figure 7 , Figure 7 Schematic diagram of the detection frame provided in the embodiment of the present application; Figure 7 Two detection boxes and the corresponding target detection results are shown in the figure. The detection result 7011 indicates that the category of the detection result of the image displayed in the detection box 701 is shoes, and the confidence result is 94.4%, indicating that there is a 94.4% probability that the image displayed in the detection box 701 is shoes. The detection result 7021 indicates that the category of the detection result of the image displayed in the detection box 702 is shoes, and the confidence result is 89.1%, indicating that there is an 89.1% probability that the image displayed in the detection box 702 is shoes.

[0125] Continue to see Figure 4 In step 404, a first regression loss is determined based on the difference between the first detection result and the true annotation box, and a second regression loss is determined based on the difference between the second detection result and the true annotation box.

[0126] As an example, based on the difference between the first detection result and the true annotation box, the first regression loss is determined. The first regression loss includes a position difference loss and a category difference loss for the detection box. The position difference loss is the position error between the detection box output by the first prediction model and the true annotation box, which is used to measure the accuracy of the first prediction model's prediction of the detection box (center point, width and height). The category difference loss is the category error between the category prediction result and the true category. The first regression loss can be implemented by formula (1), which is described in detail below.

[0127] L T =L box +L cls (1)

[0128] Among them, L T is the first regression loss (corresponding to the first model loss above), L box is the position difference loss between the detection box prediction result and the true annotation box, L cls It is the category difference loss between the category prediction result and the true category. Position difference loss (L box ) can be calculated by the intersection over union loss. The intersection over union loss (IoU Loss) directly uses the intersection over union (IoU) of the detection box and the true annotation box as the optimization target, and maximizes the overlapping area between the detection box and the true annotation box by minimizing 1-IoU. The intersection over union value range is [0, 1]. The smaller the loss value, the closer the position and size of the detection box and the true annotation box are. The category difference loss (L cls ) can be determined by calculating the cross entropy loss between the predicted category and the true category, which is not limited in this application. The second regression loss is expressed as L S (corresponding to the second model loss mentioned above), the second regression loss can be implemented by formula (2), which is described in detail below.

[0129] L T =L box +L cls (2)

[0130] The calculation method of the position difference loss and category difference loss included in the second regression loss is the same as that of the first regression loss, and will not be repeated here.

[0131] In step 405 , when the second regression loss exceeds the first regression loss, the difference between the first detection result and the second detection result is calculated to determine the difference loss.

[0132] As an example, for each detection frame, when the second regression loss exceeds the first regression loss, the difference between the first detection result and the second detection result is calculated. The difference between the detection results includes the position difference and category difference between the detection frames. The difference values ​​of all categories are accumulated to obtain the current detection frame. The differences of all detection frames are weighted and summed as the difference loss of the predicted response in the knowledge distillation process. The difference loss can be implemented by formula (3), which is explained in detail below.

[0133]

[0134] Among them, P T is the first detection result (corresponding to the first probability distribution above), that is, the output probability distribution of the first detection model, which is the predicted probability value of the target object in the current detection frame belonging to the i-th category. The probability distribution is normalized, P S is the second detection result (corresponding to the second probability distribution mentioned above), that is, the output probability distribution of the second detection model, i is the i-th category, which represents the index of all possible categories in the target detection task. Calculate the natural logarithm of the probability ratio of the first detection model and the second detection model in category i, reflecting the degree of deviation between the two predictions in this category. KL(P T ||P S ) is a logarithmically weighted sum of the ratios of the probability distribution of the first detection results and the probability distribution of the second detection results of all categories, quantifying the amount of information loss of the output distribution of the second prediction model relative to the distribution of the first prediction model.

[0135] In step 406 , the second detection model is trained based on the second regression loss and the difference loss.

[0136] As an example, the upper bound idea is a strategy to simplify the analysis or optimization process by finding the upper bound constraint of the problem. By determining the maximum possible value (upper bound) of a parameter or variable, the feasible domain of the problem is narrowed to avoid infinite exploration. When the second regression loss exceeds the first regression loss, the first regression loss (L T ) as the upper bound, define the distillation start parameter as λ, and the second regression loss (L S ) is less than the upper limit, the distillation start parameter λ=0, and only the second regression loss is optimized to avoid the first detection model knowledge interfering with its advantage part. The second regression loss (L S) is not less than the upper limit value, the difference between the second regression loss and the upper limit value is calculated, and the distillation start parameter λ is dynamically adjusted according to the size of the difference. For example, the difference threshold is set to 0.2. When the difference is less than the difference threshold, the distillation start parameter is linearly increased. When the difference is greater than the difference threshold, the distillation start parameter λ = 1. The difference loss is used to force the second detection model to learn the classification knowledge of the first detection model, thereby indirectly improving the overall detection performance.

[0137] The distillation start parameter ensures that only when the second regression loss (L S ) exceeds the first regression loss, knowledge distillation constraints are applied (only for categories). Avoiding excessive penalties helps maintain the flexibility of the second detection model. Training the second detection model based on the second regression loss and the difference loss can be achieved using formula (4), as detailed below.

[0138] L total =L S +λ·KL(P T ||P S ) (4)

[0139] Among them, L total is the total loss function for training the second detection model, L S is the second regression loss, λ is the distillation start parameter, KL(P T ||P S ) is the difference loss between the first detection model and the second detection model. Through the total loss function, it is ensured that only the second regression loss (L S ) exceeds the first regression loss, the knowledge distillation constraint is performed on it, and the second detection model is trained through the total loss function to obtain the trained lightweight second detection model.

[0140] The model training method provided in the embodiment of the present application has the following beneficial effects:

[0141] A first detection model trained on a large-scale obstacle dataset is used as the teacher model. Channels of the teacher model are pruned based on preconfigured ratios to produce a lightweight second detection model. This reduces the structure and parameters of the object detection model, alleviating system pressure. The first and second detection models analyze the differences between the first and second detection results for the same image. A knowledge distillation mechanism is triggered only when the second regression loss corresponding to the second detection model exceeds the first regression loss corresponding to the first detection model. By calculating the difference in the probability distributions of the two outputs and combining the second regression loss to construct a dynamically adjusted total loss function, this mechanism addresses the degradation of feature representation caused by model pruning. By constraining the initiation conditions of knowledge distillation through an upper bound, while retaining the second detection model's inherent predictive power, it specifically strengthens its knowledge transfer in scenarios with localization errors or insufficient classification confidence. This enables the lightweight second detection model to adaptively balance detection accuracy and computational efficiency, significantly reducing model parameter redundancy. This ensures that the second detection model can accurately identify obstacles of varying sizes and materials and output highly reliable prediction boxes and category confidence information while maintaining real-time inference speed, thereby improving the performance and efficiency of lightweight object detection models on resource-constrained devices.

[0142] The following continues to describe the exemplary structure of the model training device 455 provided in the embodiment of the present application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the model training device 455 of the memory 450 may include: a detection module 4551, used to perform model compression processing on the first target detection model to obtain a second target detection model; perform first target detection processing on the first image through the first target detection model to obtain a first target detection result, and perform second target detection processing on the first image through the second target detection model to obtain a second target detection result; a training module 4552, used to obtain the difference between the second model loss and the first model loss, and obtain a fusion parameter that is positively correlated with the difference; obtain the contrast loss between the first target detection result and the second target detection result, use the fusion parameter as the weight of the contrast loss, and fuse the second model loss and the contrast loss based on the weight to obtain a fusion loss; update the second target detection model based on the fusion loss to obtain an updated second target detection model.

[0143] In some embodiments, the detection module 4551 is also used to perform the following processing for each channel in the first target detection model: obtaining the weight parameters of the channel, and performing norm calculation processing on the weight parameters to obtain the importance of the channel; based on the importance of each channel of the first target detection model, pruning the first target detection model to obtain the second target detection model.

[0144] In some embodiments, the detection module 4551 is also used to sort the channels in descending order based on the importance of each channel to obtain a channel sorting result; filter out channels that meet a preset pruning ratio from the channel sorting result in a front-to-back order; and remove the channels filtered out in the first target detection model to obtain a second target detection model.

[0145] In some embodiments, the training module 4552 is also used to extract the first probability distribution corresponding to each candidate category under the category dimension from the first target detection result, and to extract the second probability distribution corresponding to each candidate category under the category dimension from the second target detection result; perform the following processing for each candidate category under the category dimension: perform divergence calculation processing on the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category; perform fusion processing on the contrast losses corresponding to multiple candidate categories to obtain the contrast loss between the first target detection result and the second target detection result.

[0146] In some embodiments, the training module 4552 is also used to determine the first probability distribution corresponding to the candidate category and the second probability distribution corresponding to the candidate category to perform ratio processing to obtain a first ratio; perform logarithmic calculation processing on the first ratio to obtain a first logarithmic result; multiply the first logarithmic result with the first probability distribution corresponding to the candidate category to obtain the contrast loss corresponding to the candidate category.

[0147] In some embodiments, the training module 4552 is also used to extract the first probability distribution corresponding to each candidate category in the category dimension from the first target detection result, and extract the first predicted position corresponding to the detection box in the position dimension from the first target detection result; perform difference calculation processing on the first probability distribution corresponding to each candidate category and the true probability distribution corresponding to each candidate category to obtain the loss corresponding to the category dimension; perform difference calculation processing on the first predicted position corresponding to the detection box and the first true position corresponding to the detection box to obtain the loss corresponding to the position dimension; perform fusion processing on the loss corresponding to the position dimension and the loss corresponding to the category dimension to obtain the first model loss corresponding to the first target detection result.

[0148] In some embodiments, the training module 4552 is also used to extract the second probability distribution corresponding to each candidate category in the category dimension from the second target detection result, and extract the second predicted position corresponding to the detection box in the position dimension from the second target detection result; perform difference calculation processing on the second probability distribution corresponding to each candidate category and the true probability distribution corresponding to each candidate category to obtain the loss corresponding to the category dimension; perform difference calculation processing on the second predicted position corresponding to the detection box and the second true position corresponding to the detection box to obtain the loss corresponding to the position dimension; perform fusion processing on the loss corresponding to the position dimension and the loss corresponding to the category dimension to obtain the second model loss corresponding to the second target detection result.

[0149] In some embodiments, the training module 4552 is further used to multiply the fusion parameter with the contrast loss to obtain a multiplication result, and to fuse the multiplication result with the second model loss to obtain a fusion loss.

[0150] The present invention provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the model training method described in the present invention.

[0151] The embodiment of the present application provides a computer-readable storage medium in which computer-executable instructions or computer programs are stored. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the model training method provided in the embodiment of the present application, for example, Figure 3 The model training method shown.

[0152] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0153] In some embodiments, computer-executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0154] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0155] By way of example, computer-executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed across multiple sites and interconnected by a communication network.

[0156] In summary, through the embodiments of the present application, the first target detection model is converted into a lightweight second target detection model, thereby simplifying the target detection model structure and parameters. According to the fusion parameter that is positively correlated with the difference between the first model loss and the second model loss, a contrast loss constraint is introduced to determine the fusion loss so that the second target detection model can learn the better prediction response features in the first target detection model in a targeted manner. The second target detection model is updated based on the fusion loss, and the target detection capability of the second target detection model in a complex environment is directionally enhanced by compensating the second target detection model with the knowledge transfer of the first target detection model. This allows the second target detection model to meet the lightweight requirements while maintaining detection performance and accuracy similar to that of the first target detection model.

[0157] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Performing model compression processing on the first target detection model to obtain a second target detection model; Performing a first target detection process on the first image using the first target detection model to obtain a first target detection result, and performing a second target detection process on the first image using the second target detection model to obtain a second target detection result; Obtaining a first model loss corresponding to the first object detection result, and obtaining a second model loss corresponding to the second object detection result; Obtaining a difference between the second model loss and the first model loss, and obtaining a fusion parameter positively correlated with the difference; Obtaining a contrast loss between the first target detection result and the second target detection result, taking the fusion parameter as a weight of the contrast loss, and fusing the second model loss and the contrast loss based on the weight to obtain a fusion loss; The second object detection model is updated based on the fusion loss to obtain an updated second object detection model.

2. The method according to claim 1, characterized in that The performing model compression processing on the first target detection model to obtain the second target detection model includes: The following processing is performed for each channel in the first target detection model: Obtaining a weight parameter of the channel, and performing norm calculation processing on the weight parameter to obtain the importance of the channel; Based on the importance of each channel of the first object detection model, the first object detection model is pruned to obtain the second object detection model.

3. The method according to claim 2, characterized in that The pruning of the first target detection model based on the importance of each channel of the first target detection model to obtain the second target detection model includes: Based on the importance of each channel, the channels are sorted in descending order to obtain a channel sorting result; Filter out channels that meet a preset pruning ratio from the channel sorting results in order from front to back; The channels filtered out in the first target detection model are removed to obtain the second target detection model.

4. The method according to claim 1, wherein The obtaining of the contrast loss between the first target detection result and the second target detection result includes: Extracting a first probability distribution corresponding to each candidate category under the category dimension from the first target detection result, and extracting a second probability distribution corresponding to each candidate category under the category dimension from the second target detection result; The following processing is performed for each candidate category under the category dimension: Performing divergence calculation on a first probability distribution corresponding to the candidate class and a second probability distribution corresponding to the candidate class to obtain a contrast loss corresponding to the candidate class; The contrast losses corresponding to the multiple candidate categories are fused to obtain the contrast loss between the first target detection result and the second target detection result.

5. The method according to claim 4, characterized in that The performing divergence calculation on the first probability distribution corresponding to the candidate class and the second probability distribution corresponding to the candidate class to obtain the contrast loss corresponding to the candidate class includes: performing a ratio processing on a first probability distribution corresponding to the candidate category and a second probability distribution corresponding to the candidate category to obtain a first ratio; performing logarithmic calculation processing on the first ratio to obtain a first logarithmic result; The first logarithmic result is multiplied by a first probability distribution corresponding to the candidate class to obtain a contrast loss corresponding to the candidate class.

6. The method according to claim 1, characterized in that The obtaining of a first model loss corresponding to the first target detection result includes: Extracting a first probability distribution corresponding to each candidate category in a category dimension from the first object detection result, and extracting a first predicted position corresponding to the detection box in a position dimension from the first object detection result; Performing a difference calculation process on the first probability distribution corresponding to each candidate category and the true probability distribution corresponding to each candidate category to obtain a loss corresponding to the category dimension; Performing a difference calculation process on a first predicted position corresponding to the detection frame and a first true position corresponding to the detection frame to obtain a loss corresponding to the position dimension; The loss corresponding to the position dimension and the loss corresponding to the category dimension are fused to obtain a first model loss corresponding to the first target detection result.

7. The method according to claim 1, characterized in that The obtaining of a second model loss corresponding to the second target detection result includes: Extracting a second probability distribution corresponding to each candidate category in the category dimension from the second object detection result, and extracting a second predicted position corresponding to the detection box in the position dimension from the second object detection result; Performing a difference calculation process on the second probability distribution corresponding to each candidate category and the true probability distribution corresponding to each candidate category to obtain a loss corresponding to the category dimension; Performing a difference calculation process on a second predicted position corresponding to the detection frame and a second true position corresponding to the detection frame to obtain a loss corresponding to the position dimension; The loss corresponding to the position dimension and the loss corresponding to the category dimension are fused to obtain a second model loss corresponding to the second target detection result.

8. The method according to claim 1, characterized in that The step of taking the fusion parameter as the weight of the contrast loss and fusing the second model loss and the contrast loss based on the weight to obtain a fusion loss includes: The fusion parameter is multiplied by the contrast loss to obtain a multiplication result, and the multiplication result is fused with the second model loss to obtain the fusion loss.

9. A model training device, characterized in that: The device comprises: a detection module configured to perform model compression processing on the first target detection model to obtain a second target detection model; perform first target detection processing on the first image using the first target detection model to obtain a first target detection result; and perform second target detection processing on the first image using the second target detection model to obtain a second target detection result; A training module is used to obtain the difference between the second model loss and the first model loss, and obtain a fusion parameter positively correlated with the difference; obtain the contrast loss between the first target detection result and the second target detection result, use the fusion parameter as the weight of the contrast loss, and fuse the second model loss and the contrast loss based on the weight to obtain a fusion loss; update the second target detection model based on the fusion loss to obtain an updated second target detection model.

10. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions or computer programs; The processor is used to implement the model training method described in any one of claims 1 to 8 when executing the computer-executable instructions or computer programs stored in the memory.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer-executable instructions or computer program are executed by a processor, the model training method according to any one of claims 1 to 8 is implemented.

12. A computer program product comprising computer executable instructions or a computer program, characterized in that When the computer-executable instructions or computer program are executed by a processor, the model training method according to any one of claims 1 to 8 is implemented.

Images