Image target detection method based on mask adaptive self-distillation

Abstract

The invention discloses an image target detection method based on mask adaptive self-distillation. The method comprises the following steps: constructing a target detection model; the model comprises a backbone network, a mask generation module, a classifier group, a regression network group and an adaptive weighting module; model training, including data preprocessing, feature extraction of a backbone network, mask generation, category prediction and bounding box prediction, calculation of MASD distillation loss and distillation loss sum, and optimization of model parameters by using an SGD optimizer to obtain an optimized model; wherein the backbone network, the fourth classifier and the fourth regression network are deployed as lightweight target detection models; and inputting a to-be-detected image into the optimized lightweight target detection model for target detection. According to the method disclosed by the invention, the target detection efficiency can be improved, the deployment speed is accelerated and the computing resource consumption is reduced.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image target detection method based on mask adaptive self-distillation. Background Art

[0002] With the rapid development of deep learning technology, convolutional neural networks (CNNs) have been widely used in the field of computer vision, such as target detection, image recognition, and video analysis, and have achieved remarkable results in practical application scenarios such as industrial automation. In industrial automation tasks, target detection technology requires high-precision recognition capabilities to accurately locate and classify various complex industrial parts and components, and improve the operating efficiency and automation level of production lines. For example, in assembly line automation, target detection technology helps to accurately locate and classify various parts. Based on the detection and classification feedback, the robot arm executes instructions to accurately locate and complete assembly operations; or performs some abnormal detection based on the feedback and issues early warnings, thereby improving production efficiency.

[0003] Although deep neural networks have achieved remarkable results in target detection tasks, such networks usually require large computing resources and memory, which leads to many challenges in their deployment on resource-constrained devices (such as embedded systems and industrial control terminals). Therefore, how to efficiently deploy complex deep learning models to achieve accurate target detection is one of the important issues that need to be solved in industrial automation.

[0004] In order to solve the problem of lightweight deep models, knowledge distillation technology came into being. This technology achieves the purpose of model compression by transferring knowledge from a high-performance teacher model to a smaller student model, reducing computational overhead and improving its efficiency on resource-constrained devices. In the field of target detection in industrial automation, knowledge distillation can effectively improve the real-time and accuracy of the model, thereby meeting the needs of high-precision target recognition and positioning under limited computing resources. Traditional knowledge distillation detection methods usually adopt a two-stage training process: first, train a high-capacity teacher model, and then transfer its knowledge to a smaller student model. Although this method has achieved good results, in practical applications, selecting and training complex teacher models often requires a lot of time and computing resources. In addition, traditional knowledge distillation detection methods also face challenges in the process of knowledge transfer, such as the inability to fully utilize the potential of the student model, resulting in limited effective knowledge transfer.

[0005] In order to further optimize the efficiency and performance of object detection, self-distillation technology came into being. Unlike traditional knowledge distillation detection methods, self-distillation technology allows the model to extract knowledge from itself, avoiding the need for additional teacher models. Although self-distillation technology has potential, it still faces some problems, especially in how to extract effective knowledge from the network itself. Existing self-distillation detection methods usually adopt strategies such as data augmentation, specific distribution or auxiliary branches to enhance the learning ability of the network, but these methods may lead to redundant supervision signals. When the teacher model and the student model have the same architecture and parameters, the redundant signals are highly repetitive, thus affecting the performance improvement. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a self-distillation mask adaptive image target detection method to achieve the purpose of improving target detection efficiency, accelerating deployment speed and reducing computing resource consumption.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] An image object detection method based on mask adaptive self-distillation comprises the following steps:

[0009] Step 1: Build a target detection model;

[0010] The model includes a backbone network, a mask generation module, a classifier group, a regression network group and an adaptive weighting module; the backbone network is ResNet18, including four residual blocks connected in sequence; the mask generation module includes three random mask modules and three generators, the classifier group includes four classifiers, the regression network group includes four regression networks, the first three residual blocks of the backbone network are respectively connected to the three random mask modules, and then connected to the three classifiers and the three regression networks after passing through the three generators; the fourth residual block is connected to the fourth classifier and the fourth regression network; the adaptive weighting module is respectively connected to the classifier group and the regression network group;

[0011] Step 2, model training, includes the following steps:

[0012] (1) Image data acquisition and preprocessing, and labeling on the image;

[0013] (2) The preprocessed image is input into the backbone network for feature extraction, and the four residual blocks output four feature maps respectively;

[0014] (3) The first three feature maps are input into the random mask module for mask processing to obtain the masked feature maps; the masked feature maps are reconstructed by the generator; the reconstructed feature maps and the fourth feature map are respectively input into the corresponding classifier and regression network for category prediction and bounding box prediction;

[0015] (4) The four category predictions and four bounding box predictions are self-distilled by mutual learning. During the learning process, the weights of different category predictions and bounding box predictions are adjusted according to the accuracy of each category prediction and bounding box prediction through an adaptive weighting module, and finally the MASD distillation loss is obtained;

[0016] (5) According to the obtained MASD distillation loss, the total cross entropy loss and the total regression loss are combined to calculate the total distillation loss, and the model parameters are optimized using the SGD optimizer until the model converges, and finally the optimized model is obtained; wherein the backbone network, the fourth classifier, and the fourth regression network are deployed as a lightweight target detection model;

[0017] Step 3: target detection;

[0018] The image to be detected is input into the optimized lightweight target detection model for target detection.

[0019] In the above scheme, in step (3), a random mask matrix is ​​used to perform mask processing on the feature map, as follows:

[0020] For the extracted feature map F l , where l∈(1,2,3), generates the corresponding binary mask matrix M l :

[0021]

[0022] Among them, M l is the mask matrix, m l (x,y) represents the element of the feature map at the (x,y) position. This element is a random number between 0 and 1. λ is a hyperparameter that controls the randomness of the mask.

[0023] The masked feature map F l ′Through the mask matrix M l With the original feature map F l Multiplying element by element gives:

[0024] F l ′=F l °M l

[0025] Here, ° represents the element-wise product.

[0026] In the above scheme, in step (4), the specific method of self-distilling the obtained four category prediction results by mutual learning is as follows:

[0027] (4.1) For category prediction and bounding box prediction, select one category prediction and one bounding box prediction as the current student category prediction and student bounding box prediction, and the rest as the teacher category prediction and teacher bounding box prediction;

[0028] (4.2) For teacher category prediction and teacher bounding box prediction, calculate the category weighting factor and bounding box weighting factor respectively;

[0029] (4.3) Calculate the category distillation loss and the bounding box distillation loss based on the calculated category weighting factor and bounding box weighting factor;

[0030] (4.4) According to the category distillation loss and bounding box distillation loss, let each category prediction and bounding box prediction be a student category prediction and student bounding box prediction, and calculate the MASD distillation loss.

[0031] In the above scheme, in step (4.2), for the teacher category prediction z t , first calculate the cross entropy loss:

[0032]

[0033] in, represents the cross entropy loss between the i-th teacher category prediction and the true category label, and C is the total number of categories; is the prediction value of the i-th teacher for category c, y c is the true label of category c, σ(·) is the softmax function;

[0034] Calculate teacher category prediction z t The weight of is the category weighting factor, and the specific formula is as follows:

[0035]

[0036] in, is the class weighting factor, t is the total number of teacher class predictions, is the exponential form of the cross entropy loss for the i-th category prediction.

[0037] In the above scheme, in step (4.2), for the teacher bounding box prediction b t , first calculate the regression loss, using smooth L1 loss:

[0038]

[0039] in, represents the regression loss between the i-th teacher bounding box prediction and the true bounding box, and B refers to the batch size; is the i-th teacher bounding box prediction, is the ground-truth bounding box;

[0040] Calculate weights for teacher bounding box predictions That is, the bounding box weighting factor, the specific formula is as follows:

[0041]

[0042] in, is the bounding box weighting factor, t is the total number of teacher bounding box predictions, is the exponential form of the regression loss for the i-th bounding box prediction.

[0043] In the above scheme, in step (4.3), the distillation loss L KD (z) is calculated as follows:

[0044]

[0045] Where t is the total number of teacher category predictions, is the category weighting factor, C is the total number of categories, is the prediction value of the i-th teacher for category c, z c is the current student's predicted value for category c.

[0046] In the above scheme, in step (4.3), the bounding box distillation loss L Reg (b) The calculation formula is as follows:

[0047]

[0048] Where t is the total number of teacher bounding box predictions, is the bounding box weighting factor, SmoothL1 is a regression loss function used to measure the error between the predicted value and the true value; is the i-th teacher bounding box prediction, b a is the current student bounding box prediction.

[0049] In the above scheme, in step (4.4), the MASD distillation loss L MASD The calculation is as follows:

[0050]

[0051] Among them, z l is the category prediction, l∈(1,2,3,4), z is the current student category prediction, L KD (z) represents the category distillation loss, b l is the bounding box prediction, b is the current student bounding box prediction, L Reg (b) represents the bounding box distillation loss.

[0052] In the above scheme, the total distillation loss L is calculated as follows:

[0053] L=L CE +αL Reg +βL MASD

[0054] Among them, L CE is the total cross entropy loss, L Reg is the total regression loss, L MASD is the MASD distillation loss, α and β are hyperparameters for balancing various types of losses; L CE and L Reg The calculation formula is as follows:

[0055]

[0056] Among them, L CE (z) is the cross entropy loss of the current student category prediction z, C is the total number of categories, z c is the predicted value for category c, y c is the true label of category c, σ(·) is the softmax function; L Reg (b) is the regression loss of the current student regression prediction b, B refers to the batch size, and SmoothL1 is a regression loss function used to measure the error between the predicted value and the true value; b a is the current a-th student bounding box prediction, is the true bounding box of the a-th sample.

[0057] In the above scheme, in step (5), the specific process of optimizing the model parameters using the SGD optimizer is as follows:

[0058] (5.1) Initialization parameters: All parameters in the model are randomly initialized;

[0059] (5.2) Forward propagation: Input the image, and the model outputs the detection result based on the current parameters;

[0060] (5.3) Back propagation: Calculate the gradient of the total distillation loss L with respect to the model parameters based on the detection results of the forward propagation: The gradient is the derivative of the total distillation loss L with respect to the model parameters θ. If the model parameters are θ = (θ1, θ2, …, θ n ), then the gradient It is the partial derivative of the sum of distillation losses L with respect to each parameter, forming a vector:

[0061]

[0062] (5.4) Use the SGD optimizer to update the model parameters according to the calculated gradients. In each iteration, the parameters are updated according to the following rules:

[0063]

[0064] Among them, θ t is the current parameter value, η is the learning rate;

[0065] (5.5) Repeat steps (5.2)-(5.4) until the model converges.

[0066] Through the above technical solution, the image target detection method based on mask adaptive self-distillation provided by the present invention has the following beneficial effects:

[0067] 1. Improve detection efficiency: Existing self-distillation detection methods usually have redundant supervision signals, especially when the parameters of the teacher model and the student model are the same, the learning potential of the model cannot be fully utilized. The present invention effectively reduces redundant signals and improves the expressiveness of features by introducing a mask generation module. At the same time, it combines an adaptive weighting module to dynamically adjust the weights of each layer and optimize the knowledge transfer process, thereby obtaining a target detection model with a small number of parameters and superior performance. In industrial automation tasks, the system can achieve rapid and accurate identification of target components or workpieces with fewer computing resources, significantly improve the accuracy and response speed of target detection, meet the needs of efficient production and precise identification, and ensure the accurate acquisition of key components or target information, thereby improving production efficiency and system stability.

[0068] 2. Significantly reduce computing and storage overhead: By adopting the Masked Adaptive Self Distillation (MASD) detection method, the time and computing cost of pre-training complex teacher models in traditional knowledge distillation detection methods are avoided. While ensuring the accuracy of the model, this method greatly improves the training efficiency of the lightweight model, allowing the model to be quickly deployed on resource-constrained embedded devices and quickly put into industrial automation tasks, thereby speeding up the system deployment process and reducing the development cycle.

[0069] 3. Improve model deployment efficiency: The present invention can compress the deep learning model to a sufficiently small size, reduce the computational burden and memory consumption, thereby greatly reducing the system cost and deployment complexity, and reducing the demand for computing and storage resources. In industrial automation, by deploying a lightweight self-distillation model, the system can achieve real-time monitoring on an embedded platform without relying on server-side computing resources, meet the response requirements of industrial production lines, and improve overall automation efficiency and applicability; through real-time target detection and task planning on embedded devices, the utilization of computing and storage resources is optimized, and the overall cost-effectiveness, automation efficiency and applicability of the system are improved.

[0070] 4. Enhance the adaptability of actual application scenarios: The application scenarios of the present invention include industrial automation systems. By using a lightweight deep learning model, the device can accurately identify the target and perform efficient operations. Compared with the prior art, the present invention can directly perform target detection tasks on embedded devices without relying on server-side computing resources, thereby improving the real-time performance and reliability of the system. This method is particularly suitable for intelligent manufacturing, automated detection and other fields, and has significant application value.

[0071] In general, the MASD detection method successfully solves many limitations of traditional knowledge distillation detection methods through its innovative self-distillation strategy, and provides a more efficient and streamlined deep neural network compression solution. This method can achieve rapid deployment and efficient application of deep neural networks. MASD reduces computing overhead and memory consumption by optimizing model compression, enabling industrial automation systems to achieve real-time and accurate target detection and recognition operations. This technology has shown significant application advantages in improving target detection efficiency, accelerating deployment speed, and reducing computing resource consumption, and has clear practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0073] Figure 1 A schematic diagram of a target detection model disclosed in an embodiment of the present invention.

[0074] Figure 2 This is a diagram of the model training process. DETAILED DESCRIPTION

[0075] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0076] The present invention provides an image target detection method based on mask adaptive self-distillation, comprising the following steps:

[0077] Step 1: Build a target detection model;

[0078] like Figure 1As shown, the model includes a backbone network, a mask generation module, a classifier group, a regression network group and an adaptive weighting module; the backbone network is ResNet18, including four residual blocks connected in sequence; the mask generation module includes three random mask modules and three generators, the classifier group includes four classifiers, and the regression network group includes four regression networks. The first three residual blocks of the backbone network are respectively connected to the three random mask modules, and then connected to the three classifiers and the three regression networks after passing through the three generators; the fourth residual block is connected to the fourth classifier and the fourth regression network; the adaptive weighting module is connected to the classifier group and the regression network group respectively.

[0079] The generator consists of two layers of 3x3 convolutional layers, which are designed to extract more refined feature representations from the masked feature maps. The classifier consists of one fully connected layer for category prediction, while the regression network consists of two fully connected layers for accurately predicting the bounding box of the target.

[0080] Step 2: Model training, such as Figure 2 As shown, the following steps are included:

[0081] (1) Image data acquisition and preprocessing, and labeling on the image.

[0082] In the object detection task, data acquisition and preprocessing are key steps in model training, and the quality of data directly affects the model performance. The following are the specific steps for data acquisition and preprocessing:

[0083] (1.1) Data collection

[0084] First, we collect image data containing various complex industrial parts and components from public object detection datasets. Each image should include multiple objects and their corresponding bounding boxes to indicate the location of the objects in the image. To ensure the diversity of the dataset, the collected images should cover different scenes, lighting conditions, shooting angles, and object sizes, and be suitable for visual recognition tasks in industrial automation environments.

[0085] (1.2) Data preprocessing

[0086] In the data preprocessing stage, in order to ensure that the input data can adapt to the training requirements of the model, the image data is first resized, and all images are uniformly adjusted to a fixed size to avoid errors in the training process due to inconsistent image sizes. Secondly, normalization is performed to normalize the image pixel values ​​to the interval [0,1] to improve the stability and efficiency of model training. In addition, data enhancement techniques such as random flipping, rotation, cropping, and color adjustment are used to increase the diversity of training data, enhance the robustness of the model in different scenarios, and effectively reduce the risk of overfitting. Finally, label formatting is performed to convert the target annotation information of each image into a format suitable for network training to ensure that the model can accurately process the target category and its location information. These preprocessing steps provide standardized and efficient data support for the training of subsequent models.

[0087] (1.3) Data loading

[0088] The preprocessed data is stored in the training data pipeline. The data loader feeds the data into the model based on batch B and randomly shuffles it to prevent the model from relying on the order of the data.

[0089] (2) The preprocessed image is input into the backbone network for feature extraction, and the four residual blocks output four feature maps respectively.

[0090] By extracting features of different scales through multiple convolutional layers, rich spatial and semantic information is obtained. These feature maps provide key information for subsequent classification and regression tasks.

[0091] (3) The first three feature maps are input into the random mask module for mask processing to obtain the masked feature maps; the masked feature maps are reconstructed by the generator; the reconstructed feature maps and the fourth feature map are respectively input into the corresponding classifier and regression network for category prediction and bounding box prediction.

[0092] By applying masks to feature maps at different levels, this module effectively increases the diversity of each layer's output. For the last layer of feature maps of ResNet18, no mask generation is performed, and the feature maps are directly passed to the classifier and regression network to generate prediction results. This processing method avoids masking the last layer of feature maps, ensuring that the deepest features can be fully used for the final classification and positioning tasks, and improving the prediction accuracy of the model.

[0093] The feature map is masked using a random mask matrix, as follows:

[0094] For the extracted feature map F l , where l∈(1,2,3), generates the corresponding binary mask matrix M l :

[0095]

[0096] Among them, M l is the mask matrix, m l (x, y) represents the element of the feature map at the (x, y) position. The element is a random number between 0 and 1. λ is a hyperparameter that controls the randomness of the mask. In practical applications, λ is set to 0.15.

[0097] Masked feature map F l ′Through the mask matrix M l With the original feature map F l Multiplying element by element gives:

[0098] F l ′=F l °M l

[0099] Here, ° represents the element-wise product. This operation removes redundant information and improves the effectiveness of the feature map.

[0100] Masked feature map F l ′(l∈(1,2,3)) will be sent to the corresponding generator G l (l∈(1,2,3)) performs feature reconstruction and then passes it to the corresponding classifier C l (l∈(1,2,3)) and regression network B l (l∈(1,2,3)) respectively make category prediction z l (l∈(1,2,3)) and bounding box prediction b l (l∈(1,2,3)). Finally, the model generates the category prediction z l =C l (G l (F l ′ )) and bounding box prediction b l =B l (G l (F l ′ )), where l∈(1,2,3), F l ′ is the masked feature map, G l is the generator, C l is a classifier, B l It is a regression network. For the last layer feature map F l '(l = 4), no mask generation is performed, and the feature map is directly passed to the classifier C l (l = 4) and regression network B l (l=4), used to generate the prediction result zl =C l (F l ), b l =B l (F l ). This step obtains multiple category predictions z l (l∈(1,2,3,4)) and bounding box prediction b l (l∈(1,2,3,4)).

[0101] (4) The four category predictions and four bounding box predictions are self-distilled by mutual learning. During the learning process, the weights of different category predictions and bounding box predictions are adjusted according to the accuracy of each category prediction and bounding box prediction through an adaptive weighting module, and finally the MASD distillation loss is obtained.

[0102] The main innovation of the adaptive weighting mechanism is to optimize the knowledge transfer in the distillation process by dynamically adjusting the weights between different supervision signals on the classification task and the regression task (teacher prediction as supervision). During the training process, the model not only needs to handle the category prediction task, but also needs to accurately locate the position of the target box and ensure that the model can learn valuable information from each task. The specific steps are as follows:

[0103] (4.1) For category prediction and bounding box prediction, select one of the category predictions z and one of the bounding box predictions b as the current student category prediction and student bounding box prediction, and the remaining ones as the teacher category prediction z t and the teacher bounding box prediction b t .

[0104] (4.2) For teacher category prediction and teacher bounding box prediction, the category weighting factor and bounding box weighting factor are calculated respectively.

[0105] For the teacher category prediction z t , first calculate the cross entropy loss, which is used to measure the error in category classification:

[0106]

[0107] in, represents the cross entropy loss between the i-th teacher category prediction and the true category label, and C is the total number of categories; is the prediction value of the i-th teacher for category c, y c is the true label of category c, usually encoded using one-hot encoding, and σ(·) is the softmax function;

[0108] Calculate the prediction z for each teacher category t The weight of is the category weighting factor, and the specific formula is as follows:

[0109]

[0110] in, is the class weighting factor, t is the total number of teacher class predictions, is the exponential form of the cross entropy loss for the i-th category prediction. The purpose of this is to increase the weight corresponding to small losses (i.e. more confident predictions). Calculation When , a smaller cross entropy loss (i.e., higher confidence) leads to a larger weight, and vice versa.

[0111] For the teacher bounding box prediction b t , first calculate the regression loss, which is used to calculate the predicted bounding box and the ground-truth bounding box The difference between , this part is calculated using smooth L1 loss:

[0112]

[0113] in, represents the regression loss between the i-th teacher bounding box prediction and the true bounding box, and B refers to the batch size; is the i-th teacher bounding box prediction, is the ground-truth bounding box;

[0114] Calculate the weights for each teacher's bounding box prediction That is, the bounding box weighting factor, the specific formula is as follows:

[0115]

[0116] in, is the bounding box weighting factor, t is the total number of teacher bounding box predictions, is the exponential form of the regression loss for the i-th bounding box prediction. The purpose of this is to increase the weight corresponding to small losses (i.e. more confident predictions). Calculation When , a smaller regression loss (i.e., higher confidence) leads to a larger weight, and vice versa.

[0117] The purpose of this weighting mechanism is to make the network pay more attention to those bounding box predictions with higher confidence, so that the network can learn from more reliable supervision signals.

[0118] (4.3) Calculate the category distillation loss and the bounding box distillation loss based on the calculated category weighting factor and bounding box weighting factor;

[0119] Category Distillation Loss L KD (z) By weighted summing of category predictions, the distillation effect of the model on high confidence categories is improved. The calculation formula is as follows:

[0120]

[0121] Where t is the total number of teacher category predictions, is the category weighting factor, C is the total number of categories, is the prediction value of the i-th teacher for category c, z c is the current student's predicted value for category c.

[0122] Bounding box distillation loss L Reg (b) Calculated by weighted regression loss, the calculation formula is as follows:

[0123]

[0124] Where t is the total number of teacher bounding box predictions, is the bounding box weighting factor, SmoothL1 is a regression loss function used to measure the error between the predicted value and the true value; is the i-th teacher bounding box prediction, b a is the current student bounding box prediction. This allows the network to dynamically weight different outputs and focus on more confident outputs for knowledge distillation.

[0125] (4.4) According to the category distillation loss and bounding box distillation loss, let each category prediction and bounding box prediction be a student category prediction and student bounding box prediction, and calculate the MASD distillation loss.

[0126] For any output z belonging to z l and b belongs to b l , both the category distillation loss and the bounding box distillation loss need to be calculated, and the MASD distillation loss L MASD The calculation is as follows:

[0127]

[0128] Among them, z l is the category prediction, l∈(1,2,3,4), z is the current student category prediction, L KD (z) represents the category distillation loss, b l is the bounding box prediction, b is the current student bounding box prediction, L Reg (b) represents the bounding box distillation loss.

[0129] (5) Based on the obtained MASD distillation loss, the total cross entropy loss and the total regression loss are combined to calculate the total distillation loss, and the SGD optimizer is used to optimize the model parameters until the model converges, and finally the optimized model is obtained; among them, the backbone network, the fourth classifier, and the fourth regression network are deployed as a lightweight target detection model.

[0130] The total distillation loss L is calculated as follows:

[0131] L=L CE +aL Reg +βL MAsD

[0132] Among them, L CE is the total cross entropy loss, L Reg is the total regression loss, L MASD is the MASD distillation loss, and a and β are hyperparameters that balance various types of losses.

[0133] L CE and L Reg It is calculated based on the real label, and the calculation formula is as follows:

[0134]

[0135]

[0136] Among them, L CE (z) is the cross entropy loss of the current student category prediction z, C is the total number of categories, z c is the predicted value for category c, y c is the true label of category c, σ(·) is the softmax function; L Reg (b) is the regression loss of the current student regression prediction b, B refers to the batch size, and SmoothL1 is a regression loss function used to measure the error between the predicted value and the true value; b a is the current a-th student bounding box prediction, is the true bounding box of the a-th sample.

[0137] The specific process of optimizing model parameters using SGD optimizer is as follows:

[0138] (5.1) Initialization parameters: All parameters in the model are randomly initialized;

[0139] (5.2) Forward propagation: Input the image, and the model outputs the detection result based on the current parameters;

[0140] (5.3) Back propagation: Calculate the gradient of the total distillation loss L with respect to the model parameters based on the detection results of the forward propagation: The gradient is the derivative of the total distillation loss L with respect to the model parameters θ. If the model parameters are θ = (θ1, θ2, …, θ n ), then the gradient It is the partial derivative of the sum of distillation losses L with respect to each parameter, forming a vector:

[0141]

[0142] (5.4) Use the SGD optimizer to update the model parameters according to the calculated gradients. In each iteration, the parameters are updated according to the following rules:

[0143]

[0144] Among them, θ t is the current parameter value, η is the learning rate;

[0145] (5.5) Repeat steps (5.2)-(5.4) until the model converges.

[0146] The entire training process uses the SGD optimizer with a momentum of 0.9 to accelerate convergence and reduce oscillation. The batch size is set to 128 and the weight decay is 1×10 -4 . A linear warm-up phase of 5 epochs was set at the beginning of training, and the learning rate was gradually increased from 0 to 0.1 to avoid model collapse during self-distillation training. Subsequently, during the training process of a total of 200 epochs, the learning rate was reduced by 10 times when reaching the 100th and 150th epochs, respectively, to achieve more detailed parameter optimization in later training. The entire training process includes initialization, linear warm-up, main training phase and parameter update. In each batch, the model parameters are updated using the SGD optimizer according to the current learning rate and momentum, and regularization is continuously performed through the weight decay mechanism to ensure the stability and generalization ability of the model on different datasets. In addition, during the optimization process, the backbone network is trained in conjunction with the generator, classifier, and regression network to improve the detection performance of the overall model. However, in actual deployment, only the optimized backbone network and the classifier and regression network that process unmasked features are retained to achieve efficient model inference and resource utilization, ensuring real-time and accurate object detection on resource-constrained devices.

[0147] In terms of hyperparameter selection, the initial learning rate is set to 0.001, and a decay strategy is adopted to reduce the learning rate by 10% after each round. In order to prevent overfitting, L2 regularization (weight decay) is used, and the decay value is set to 1×10 -4In addition, the hyperparameters α and β were set to 1 and 1.5, respectively, and these hyperparameters were further optimized through cross-validation to improve the performance and generalization ability of the model.

[0148] Step 3: target detection;

[0149] The image to be detected is input into the optimized lightweight target detection model for target detection.

[0150] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for image object detection based on mask adaptive self-distillation, characterized in that: The steps include: Step 1: Build a target detection model; The model includes a backbone network, a mask generation module, a classifier group, a regression network group and an adaptive weighting module; the backbone network is ResNet18, including four residual blocks connected in sequence; the mask generation module includes three random mask modules and three generators, the classifier group includes four classifiers, the regression network group includes four regression networks, the first three residual blocks of the backbone network are respectively connected to the three random mask modules, and then connected to the three classifiers and the three regression networks after passing through the three generators; the fourth residual block is connected to the fourth classifier and the fourth regression network; the adaptive weighting module is respectively connected to the classifier group and the regression network group; Step 2, model training, includes the following steps: (1) Image data acquisition and preprocessing, and labeling on the image; (2) The preprocessed image is input into the backbone network for feature extraction, and the four residual blocks output four feature maps respectively; (3) Input the first three feature maps into the random mask module for masking to obtain the masked feature maps; The masked feature map is reconstructed by the generator; the reconstructed feature map and the fourth feature map are input into the corresponding classifier and regression network respectively for category prediction and bounding box prediction; (4) The four category predictions and four bounding box predictions are self-distilled by mutual learning. During the learning process, the weights of different category predictions and bounding box predictions are adjusted according to the accuracy of each category prediction and bounding box prediction through an adaptive weighting module, and finally the MASD distillation loss is obtained; (5) According to the obtained MASD distillation loss, the total cross entropy loss and the total regression loss are combined to calculate the total distillation loss, and the model parameters are optimized using the SGD optimizer until the model converges, and finally the optimized model is obtained; wherein the backbone network, the fourth classifier, and the fourth regression network are deployed as a lightweight target detection model; Step 3: target detection; The image to be detected is input into the optimized lightweight target detection model for target detection.

2. The image object detection method based on mask adaptive self-distillation according to claim 1, characterized in that: In step (3), a random mask matrix is ​​used to mask the feature map, as follows: For the extracted feature map F l , where l∈(1,2,3), generates the corresponding binary mask matrix M l : Among them, M l is the mask matrix, m l (x,y) represents the element of the feature map at the (x,y) position. This element is a random number between 0 and 1. λ is a hyperparameter that controls the randomness of the mask. Masked feature map F l ′Through the mask matrix M l With the original feature map F l Multiplying element by element gives: in, Stands for element-wise product.

3. The image object detection method based on mask adaptive self-distillation according to claim 1, characterized in that: In step (4), the specific method of self-distilling the obtained four category prediction results by mutual learning is as follows: (4.1) For category prediction and bounding box prediction, select one category prediction and one bounding box prediction as the current student category prediction and student bounding box prediction, and the rest as the teacher category prediction and teacher bounding box prediction; (4.2) For teacher category prediction and teacher bounding box prediction, calculate the category weighting factor and bounding box weighting factor respectively; (4.3) Calculate the category distillation loss and the bounding box distillation loss based on the calculated category weighting factor and bounding box weighting factor; (4.4) According to the category distillation loss and bounding box distillation loss, let each category prediction and bounding box prediction be a student category prediction and student bounding box prediction, and calculate the MASD distillation loss.

4. The image target detection method based on mask adaptive self-distillation according to claim 3, characterized in that: In step (4.2), for the teacher category prediction z t , first calculate the cross entropy loss: in, represents the cross entropy loss between the i-th teacher category prediction and the true category label, and C is the total number of categories; is the prediction value of the i-th teacher for category c, y c is the true label of category c, σ(·) is the softmax function; Calculate teacher category prediction z t The weight of is the category weighting factor, and the specific formula is as follows: in, is the class weighting factor, t is the total number of teacher class predictions, is the exponential form of the cross entropy loss for the i-th category prediction.

5. The image object detection method based on mask adaptive self-distillation according to claim 3, characterized in that: In step (4.2), for the teacher bounding box prediction b t , first calculate the regression loss, using smooth L1 loss: in, represents the regression loss between the i-th teacher bounding box prediction and the true bounding box, and B refers to the batch size; is the i-th teacher bounding box prediction, is the ground-truth bounding box; Calculate weights for teacher bounding box predictions That is, the bounding box weighting factor, the specific formula is as follows: in, is the bounding box weighting factor, t is the total number of teacher bounding box predictions, is the exponential form of the regression loss for the i-th bounding box prediction.

6. The image object detection method based on mask adaptive self-distillation according to claim 3, characterized in that: In step (4.3), the category distillation loss L KD (z) is calculated as follows: Where t is the total number of teacher category predictions, is the category weighting factor, C is the total number of categories, is the prediction value of the i-th teacher for category c, z c is the current student's predicted value for category c.

7. The image object detection method based on mask adaptive self-distillation according to claim 3, characterized in that: In step (4.3), the bounding box distillation loss L Reg (b) The calculation formula is as follows: Where t is the total number of teacher bounding box predictions, is the bounding box weighting factor, SmoothL1 is a regression loss function used to measure the error between the predicted value and the true value; is the i-th teacher bounding box prediction, b a is the current student bounding box prediction.

8. The image object detection method based on mask adaptive self-distillation according to claim 3, characterized in that: In step (4.4), the MASD distillation loss L MASD The calculation is as follows: Among them, z l is the category prediction, l∈(1,2,3,4), z is the current student category prediction, L KD (z) represents the category distillation loss, b l is the bounding box prediction, b is the current student bounding box prediction, L Reg (b) represents the bounding box distillation loss.

9. The image object detection method based on mask adaptive self-distillation according to claim 1, characterized in that: The total distillation loss L is calculated as follows: L=L CE +αL Reg +βL MAsD Among them, L CE is the total cross entropy loss, L Reg is the total regression loss, L MASD is the MASD distillation loss, α and β are hyperparameters for balancing various types of losses; L CE and L Reg The calculation formula is as follows: Among them, L CE (z) is the cross entropy loss of the current student category prediction z, C is the total number of categories, z c is the predicted value for category c, y c is the true label of category c, σ(·) is the softmax function; L Reg (b) is the regression loss of the current student regression prediction b, B refers to the batch size, and SmoothL1 is a regression loss function used to measure the error between the predicted value and the true value; b a is the current a-th student bounding box prediction, is the true bounding box of the a-th sample.

10. The image object detection method based on mask adaptive self-distillation according to claim 9, characterized in that: In step (5), the specific process of optimizing model parameters using SGD optimizer is as follows: (5.1) Initialization parameters: All parameters in the model are randomly initialized; (5.2) Forward propagation: Input the image, and the model outputs the detection result based on the current parameters; (5.3) Back propagation: Calculate the gradient of the total distillation loss L with respect to the model parameters based on the detection results of the forward propagation: The gradient is the derivative of the total distillation loss L with respect to the model parameters θ. If the model parameters are θ = (θ1, θ2, …, θ n ), then the gradient It is the partial derivative of the sum of distillation losses L with respect to each parameter, forming a vector: (5.4) Use the SGD optimizer to update the model parameters according to the calculated gradients. In each iteration, the parameters are updated according to the following rules: Among them, θ t is the current parameter value, η is the learning rate; (5.5) Repeat steps (5.2)-(5.4) until the model converges.

Images