Target detection optimization method, device and equipment based on two-dimensional knowledge distillation improvement

Abstract

The invention relates to the technical field of detection, in particular to a target detection optimization method, device and equipment based on two-dimensional knowledge distillation improvement, and the method comprises the steps: obtaining a mixed illumination image set; training a pre-constructed teacher model by using the mixed illumination image set to obtain a training result, and guiding a pre-constructed student model to perform offline knowledge distillation based on the training result; performing online knowledge distillation on the mixed illumination image set by using the trained teacher model and the student model after offline distillation to generate a target detection model, and obtaining a to-be-detected image; and extracting brightness features of the to-be-detected image, loading model parameters of the target detection model according to the brightness features, and detecting the to-be-detected image by using the loaded target detection model. Therefore, the problems of limited adaptability to different illumination and environments, poor detection precision, poor stability and the like of a knowledge distillation method in related technologies are solved.

Description

Technical Field

[0001] This invention relates to the field of detection technology, and in particular to a target detection optimization method, apparatus and equipment based on two-dimensional knowledge distillation. Background Technology

[0002] Due to drastic changes in lighting conditions, weather, and background scenes, the same type of target exhibits significant visual feature differences under different acquisition environments. Knowledge distillation, as an effective model transfer and performance enhancement technique, transfers the soft labels or intermediate feature knowledge output by the teacher model to the student model, significantly improving detection accuracy and robustness while maintaining model lightweightness. However, knowledge distillation methods in related technologies mostly focus on model compression or performance improvement within the same domain, lacking the ability to model feature structure differences across lighting domains. Their adaptability to different lighting and environments is limited, especially in scenes with day-night cycles, changes in weather, indoor-outdoor transitions, and complex sea and underwater environments, where image brightness, contrast, color distribution, and texture clarity change significantly, leading to a substantial decrease in detection accuracy. Summary of the Invention

[0003] This invention provides a target detection optimization method, apparatus, electronic device, and storage medium based on two-dimensional knowledge distillation, to solve the problems of limited adaptability of knowledge distillation methods in related technologies to different lighting and environments, as well as poor detection accuracy and poor stability.

[0004] A first aspect of the present invention provides a target detection optimization method based on two-dimensional knowledge distillation, comprising the following steps: acquiring a mixed illumination image set; training a pre-constructed teacher model using the mixed illumination image set to obtain training results, and using the training results to guide a pre-constructed student model to perform offline knowledge distillation; using the trained teacher model and the offline distilled student model to perform online knowledge distillation on the mixed illumination image set to generate a target detection model and acquire the image to be detected; extracting the brightness features of the image to be detected, loading the model parameters of the target detection model according to the brightness features, and using the loaded target detection model to detect the image to be detected.

[0005] Optionally, an online knowledge distillation is performed on the mixed-light image set using the trained teacher model and the offline distilled student model to generate a target detection model, including: dividing the mixed-light image set into a daytime image subset and a nighttime image subset using a classification model; performing nighttime online knowledge distillation on the nighttime image subset based on the pre-trained teacher model and the offline distilled student model to accumulate a nighttime optimized parameter set, and performing daytime online knowledge distillation on the daytime image subset to accumulate a daytime optimized parameter set; fusing the nighttime optimized parameter set to generate a first target model parameter set suitable for nighttime scenes, and fusing the daytime optimized parameter set to generate a second target model parameter set suitable for daytime scenes; and integrating the first target model parameter set and the second target model parameter set into the offline distilled student model to obtain a target detection model.

[0006] Optionally, the brightness features include at least one of the following: image grayscale mean, brightness channel mean, brightness histogram distribution, and exposure level. The model parameters of the target detection model are loaded based on the brightness features, and the loaded target detection model is used to detect the image to be detected. This includes: determining the actual scene of the image to be detected based on the brightness features; if the actual scene is a night scene, then loading the first target model parameters to detect the image to be detected; if the actual scene is a daytime scene, then loading the second target model parameters to detect the image to be detected.

[0007] Optionally, the mixed illumination images are divided into daytime image subsets and nighttime image subsets using a classification model, including: inputting the mixed illumination image set into a first classification model and a second classification model in sequence, wherein the first classification model outputs seasonal data for each image in the mixed illumination image set, and the second classification model outputs temporal data for each image in the mixed illumination image set; and dividing the mixed illumination image set into daytime image subsets and nighttime image subsets based on the seasonal data and temporal data.

[0008] Optionally, offline knowledge distillation employs a temperature-weighted soft-label distillation strategy to minimize the total loss function of the pre-built student model during the offline knowledge distillation process, thereby obtaining the offline distilled student model.

[0009] Optionally, a nighttime online knowledge distillation accumulation is performed on a subset of nighttime images based on a pre-trained teacher model and an offline-distilled student model to obtain a set of nighttime optimized parameters. This includes: acquiring each nighttime image in the nighttime image subset; using the teacher model and the offline-distilled student model to predict each nighttime image sequentially, calculating the distillation loss value corresponding to each nighttime image, and storing it in a pre-built cache until the cache meets preset conditions; calculating the average loss value of all distillation loss values; and using the average loss value to backpropagate and update the student model parameters to obtain the nighttime optimized parameters.

[0010] Optionally, based on a pre-trained teacher model and an offline-distilled student model, an online knowledge distillation process is performed on a subset of daytime images to optimize the daytime parameter set. This includes: acquiring each daytime image in the subset of daytime images; using the teacher model and the offline-distilled student model to predict each daytime image sequentially, calculating the distillation loss value corresponding to each daytime image, and storing it in a pre-built cache until the cache meets preset conditions; calculating the average loss value of all distillation loss values; and using the average loss value to backpropagate and update the student model parameters to obtain the daytime optimization parameters.

[0011] A second aspect of the present invention provides an optimized target detection apparatus based on two-dimensional knowledge distillation, comprising: an acquisition module for acquiring a mixed-illumination image set; a training module for training a pre-constructed teacher model using the mixed-illumination image set to obtain training results, and using the training results to guide a pre-constructed student model to perform offline knowledge distillation; a generation module for performing online knowledge distillation on the mixed-illumination image set using the trained teacher model and the offline-distilled student model to generate a target detection model and acquire an image to be detected; and a detection module for extracting brightness features of the image to be detected, loading model parameters of the target detection model according to the brightness features, and using the loaded target detection model to detect the image to be detected.

[0012] Optionally, the generation module is further configured to: divide the mixed illumination image set into a daytime image subset and a nighttime image subset using a classification model; perform nighttime online knowledge distillation on the nighttime image subset based on a pre-trained teacher model and an offline distilled student model to accumulate a nighttime optimized parameter set, and perform daytime online knowledge distillation on the daytime image subset to optimize the daytime parameter set; fuse the nighttime optimized parameter set to generate a first target model parameter set suitable for nighttime scenes, fuse the daytime optimized parameter set to generate a second target model parameter set suitable for daytime scenes; and integrate the first target model parameter set and the second target model parameter set into the offline distilled student model to obtain a target detection model.

[0013] Optionally, the brightness features include at least one of the image grayscale mean, brightness channel mean, brightness histogram distribution, and exposure level. The detection module is further used to determine the actual scene of the image to be detected based on the brightness features. If the actual scene is a night scene, the first target model parameters are loaded to detect the image to be detected. If the actual scene is a daytime scene, the second target model parameters are loaded to detect the image to be detected.

[0014] Optionally, the generation module is further used to input the mixed illumination image set into the first classification model and the second classification model in sequence, wherein the first classification model outputs the seasonal data of each image in the mixed illumination image set, and the second classification model outputs the temporal data of each image in the mixed illumination image set; based on the seasonal data and temporal data, the mixed illumination image set is divided into a daytime image subset and a nighttime image subset.

[0015] Optionally, offline knowledge distillation employs a temperature-weighted soft-label distillation strategy to minimize the total loss function of the pre-built student model during the offline knowledge distillation process, thereby obtaining the offline distilled student model.

[0016] Optionally, the generation module is further used to acquire each night image in the night image subset; use the teacher model and the offline distilled student model to predict and calculate the distillation loss value corresponding to each night image in turn, and store it in a pre-built buffer until the buffer meets the preset conditions, and calculate the average loss value of all distillation loss values; use the average loss value to backpropagate and update the student model parameters to obtain the night optimization parameters.

[0017] Optionally, the generation module is further used to acquire each daytime image in the daytime image subset; use the teacher model and the offline distilled student model to predict and calculate the distillation loss value corresponding to each daytime image in turn, and store it in a pre-built cache until the cache meets the preset conditions, and calculate the average loss value of all distillation loss values; use the average loss value to backpropagate and update the student model parameters to obtain the daytime optimization parameters.

[0018] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target detection optimization method based on two-dimensional knowledge distillation as described in the above embodiments.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed, are used to implement the target detection optimization method based on two-dimensional knowledge distillation as described in the above embodiments.

[0020] Therefore, the present invention has at least the following beneficial effects: This invention employs a dual knowledge distillation mechanism combining offline and online methods on a mixed-light image set to generate specialized model parameters suitable for nighttime and daytime scenes. These parameters are then integrated into a student model to construct the final target detection model. During the detection phase, the corresponding specialized model parameters are dynamically switched and loaded based on the brightness characteristics of the image under test, thereby enabling the model to adapt to different lighting environments. Through scene-specific refined optimization, the robustness and accuracy of the target model under different lighting conditions during the day and night are effectively improved, mitigating the performance degradation caused by lighting differences. This achieves stable and efficient target detection capabilities in all weather conditions, solving the problems of limited adaptability of knowledge distillation methods to different lighting and environments, as well as poor detection accuracy and stability.

[0021] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0022] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart of a target detection optimization method based on two-dimensional knowledge distillation provided according to an embodiment of the present invention; Figure 2 This is a schematic diagram of dataset partitioning and offline knowledge distillation according to an embodiment of the present invention; Figure 3 A schematic diagram of online knowledge distillation provided according to an embodiment of the present invention; Figure 4 This is a schematic diagram of optimal parameter fusion provided according to an embodiment of the present invention; Figure 5 This is a block diagram of a target detection optimization device based on two-dimensional knowledge distillation provided according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0024] The following describes, with reference to the accompanying drawings, an optimization method, apparatus, electronic device, and storage medium for target detection based on two-dimensional knowledge distillation according to embodiments of the present invention. Addressing the problems mentioned in the background art, the present invention provides an optimization method for target detection based on two-dimensional knowledge distillation. In this method, a dual knowledge distillation mechanism combining offline and online methods is employed on a mixed-light image set to generate specialized model parameters suitable for nighttime and daytime scenes, which are then integrated into a student model to construct the final target detection model. During the detection phase, the corresponding specialized model parameters are dynamically switched and loaded according to the brightness characteristics of the image to be tested, thereby achieving model adaptation to different lighting environments. Thus, through scene-specific refined optimization, the robustness and accuracy of the target model under different lighting conditions during the day and night are effectively improved, alleviating the performance degradation problem caused by lighting differences. This achieves stable and efficient target detection capabilities in all weather conditions, solving the problems of limited adaptability of related knowledge distillation methods to different lighting and environments, and poor detection accuracy and stability.

[0025] Specifically, Figure 1 This is a flowchart illustrating a target detection optimization method based on two-dimensional knowledge distillation, provided in an embodiment of the present invention.

[0026] like Figure 1 As shown, the target detection optimization method based on two-dimensional knowledge distillation includes the following steps: In step S101, a set of mixed illumination images is obtained.

[0027] In this embodiment of the invention, the mixed lighting image set is a mixed image set without day and night labels, which naturally covers a variety of lighting scenes from sunny daytime to dark nighttime.

[0028] In step S102, a pre-built teacher model is trained using a mixed illumination image set to obtain training results, and the training results are used to guide a pre-built student model to perform offline knowledge distillation.

[0029] In the offline knowledge distillation process, a temperature-weighted soft-label distillation strategy is used to obtain the offline distilled student model from the total loss function of the pre-built student model.

[0030] Specifically, in this embodiment of the invention, a complex and high-performance teacher model is first trained on the original image dataset that does not distinguish between day and night lighting conditions. The training process is represented as follows: ,in, This represents the loss function of the teacher model. Let cross-entropy be the loss function. For real labels, This is the predicted output of the teacher model. Then, a student model with a simple structure and few parameters is trained, with the loss function being: , It is the total loss function of the student model. To control the balance between the loss weights of hard labels and soft labels, This is the predicted output of the student model. This is the knowledge distillation loss function, used to measure the similarity between the outputs of the teacher model and the student model. The purpose of knowledge distillation is to enable the student model to learn the soft labels of the teacher model, i.e., the predicted probability distribution of the teacher model. The calculation formula is: Where T is a temperature parameter used to control the smoothness of the soft labels, and the final result of training the student model is to minimize the total loss function of the student model, i.e., minθ. max After training, the performance of the student model is evaluated to ensure that it achieves high accuracy while maintaining efficiency. The loss function for the offline knowledge distillation process is defined as follows:

[0031] For cross-entropy loss, KL is the Kullback-Leibler divergence. For two probability distributions P and Q, the KL divergence is calculated as follows: KL(P||Q) = ,in, Let represent the probability of the i-th class output of the teacher model after Softmax normalization at temperature T. This represents the corresponding probability of the student model. The KL divergence is the softmax function; the smaller the divergence, the better the student model fits the predicted distribution of the teacher model. T is the temperature coefficient. , As a weighting factor, the output of this stage is used to initialize the student model parameters. And back up the teacher model in the cloud.

[0032] In step S103, the trained teacher model and the offline distilled student model are used to perform online knowledge distillation on the mixed illumination image set to generate a target detection model and obtain the image to be detected.

[0033] Among them, the images to be detected are unknown images that need to be identified as debris and captured in real time by actual marine monitoring equipment (such as drones, underwater robots, and coastal cameras).

[0034] In one embodiment of the present invention, an online knowledge distillation is performed on a set of mixed-light images using a trained teacher model and an offline distilled student model to generate a target detection model. This includes: dividing the mixed-light images into a daytime image subset and a nighttime image subset using a classification model; performing nighttime online knowledge distillation on the nighttime image subset based on the pre-trained teacher model and the offline distilled student model to accumulate a nighttime optimized parameter set, and performing daytime online knowledge distillation on the daytime image subset to accumulate a daytime optimized parameter set; fusing the nighttime optimized parameter set to generate a first target model parameter set suitable for nighttime scenes, and fusing the daytime optimized parameter set to generate a second target model parameter set suitable for daytime scenes; and integrating the first and second target model parameters into the offline distilled student model to obtain a target detection model.

[0035] It is understood that the embodiments of the present invention divide the mixed lighting image set into daytime and nighttime subsets using a classification model, and perform online knowledge distillation on the two subsets respectively to accumulate an optimized parameter set for different lighting conditions. Then, the optimized parameter set is fused to generate a special model parameter suitable for nighttime and daytime scenes, and integrated into the offline distilled student model to build the final target detection model. This takes into account the actual situation of drastic changes in lighting and significant domain shifts in complex environments (such as marine environments). Through scene-specific fine optimization, the robustness and accuracy of the student model under different lighting conditions during the day and night are effectively improved, the performance degradation caused by lighting differences is alleviated, and a stable and efficient target detection capability is achieved in all weather conditions.

[0036] In one embodiment of the present invention, a classification model is used to divide mixed lighting images into a daytime image subset and a nighttime image subset, including: inputting the mixed lighting image set into a first classification model and a second classification model in sequence, wherein the first classification model outputs seasonal data for each image in the mixed lighting image set, and the second classification model outputs temporal data for each image in the mixed lighting image set; and dividing the mixed lighting image set into a daytime image subset and a nighttime image subset based on the seasonal data and temporal data.

[0037] Specifically, in this embodiment of the invention, a CNN classification model can be trained first using the Fomo dataset to identify the season based on the image capture time, resulting in the first CNN classification model. Then, the BDD100k dataset is used to train another CNN model, enabling it to distinguish specific time periods within a day, resulting in the second CNN classification model. The original Trash-ICRA19 dataset is processed sequentially through the first and second CNN classification models to obtain two data subsets: daytime dataset A and nighttime dataset B, which serve as the input basis for subsequent online knowledge distillation.

[0038] In one embodiment of the present invention, a nighttime online knowledge distillation accumulation is performed on a subset of nighttime images based on a pre-trained teacher model and an offline distilled student model to obtain a nighttime optimization parameter set. This includes: acquiring each nighttime image in the nighttime image subset; using the teacher model and the offline distilled student model to predict each nighttime image sequentially, calculating the distillation loss value corresponding to each nighttime image, and storing it in a pre-built cache until the cache meets preset conditions; calculating the average loss value of all distillation loss values; and using the average loss value to backpropagate and update the student model parameters to obtain the nighttime optimization parameters.

[0039] The default condition is that the buffer is filled.

[0040] Understandably, since the teacher model and the offline-distilled student model have stronger daytime image recognition capabilities, the nighttime image set B is first used for online distillation to improve the model's recognition ability in low-light environments. Specifically, this embodiment of the invention can sequentially output the nighttime images in subset B. For each image processed, the cloud-based teacher model and student model make predictions on it, and calculate the distillation loss of the prediction. The loss will be calculated after each calculation. Store the data in the buffer until it is full (N can be preset to 32), then average all losses. Then, using average loss Backpropagation updates the student model parameters; this process is repeated 50 times to obtain the optimized nighttime parameters. Store these parameters in the cloud for later fusion and retrieval. After multiple rounds of parameter iterations and updates, the student model now possesses good nighttime target detection and recognition capabilities.

[0041] It should be noted that once the average loss has been used to update the model, the 32 original, individual loss values ​​have served their purpose and are no longer needed. To prepare for the next round of 32 new images, all old data in the buffer must be cleared to make room for the newly calculated loss values, thereby progressively optimizing the student model's parameters. Secondly, the online distillation mechanism based on buffer caching in this embodiment significantly reduces the impact of low-quality image pseudo-label error propagation on the model. Traditional self-training methods rely on initial pseudo-labels, which easily lead to error accumulation. This invention averages the distillation loss in each round before backpropagation, improving the stability and effectiveness of the optimization process.

[0042] In one embodiment of the present invention, daytime online knowledge distillation of daytime optimization parameter sets is performed on a subset of daytime images based on a pre-trained teacher model and an offline distilled student model. This includes: acquiring each daytime image in the daytime image subset; using the teacher model and the offline distilled student model to predict each daytime image sequentially, calculating the distillation loss value corresponding to each daytime image, and storing it in a pre-built cache until the cache meets preset conditions; calculating the average loss value of all distillation loss values; and using the average loss value to backpropagate and update the student model parameters to obtain the daytime optimization parameters.

[0043] Understandably, during the daytime, the student model's parameters have been modified to perform better in target detection at night, resulting in weaker daytime target detection capabilities. Therefore, online knowledge distillation during the daytime is necessary. Using a daytime image dataset A as input, the cloud-based teacher and student models are used to predict and calculate losses for the images, respectively. The distillation loss is written to a buffer, and the average value is calculated when the buffer capacity reaches N. The student model parameters were then optimized using this loss, and 50 rounds of iterative training were performed to obtain the daytime optimized student model parameters. The data is stored in the cloud for easy integration and reuse later. After multiple rounds of parameter iteration and updates, the student model then possesses good daytime target detection and recognition capabilities.

[0044] Furthermore, this embodiment of the invention performs multiple rounds of alternating online knowledge distillation and parameter fusion. The online knowledge distillation process is executed alternately between night and day for a total of M=1000 rounds, generating student model parameters for multiple rounds: Nighttime parameter set: { }, Daytime parameter set: { }, take the average value for each set of parameters: , This process yields first target model parameters suitable for nighttime scenarios and second target model parameters suitable for daytime scenarios, which are then integrated into the student model. Thus, a multi-round day-night alternating distillation strategy achieves a balanced improvement in detection capabilities for both daytime and nighttime conditions, and a parameter fusion method generates optimal detection model parameters for each of the two illumination conditions. In subsequent applications, this embodiment of the invention can dynamically switch student model parameters to achieve illumination domain adaptation of the target detection model, automatically loading optimal parameters (daytime or nighttime parameters), significantly improving the stability and accuracy of all-weather target recognition.

[0045] In step S104, the brightness features of the image to be detected are extracted, the model parameters of the target detection model are loaded according to the brightness features, and the loaded target detection model is used to detect the image to be detected.

[0046] The brightness features include at least one of the following: image grayscale mean, brightness channel mean, brightness histogram distribution, and exposure level.

[0047] In one embodiment of the present invention, the model parameters of the target detection model are loaded according to the brightness features, and the loaded target detection model is used to detect the image to be detected, including: determining the actual scene of the input image according to the brightness features; if the actual scene is a night scene, then loading the first target model parameters to detect the image to be detected; if the actual scene is a daytime scene, then loading the second target model parameters to detect the image to be detected.

[0048] Understandably, features such as the mean of the image brightness histogram and HSV brightness distribution are used to determine the current lighting state of the image. If it is a nighttime scene, then the first target parameter is loaded. If it is a daytime scene, then load the second target parameter. This significantly enhances the model's ability to stably identify targets in both daytime and nighttime images, and solves the problem of the decline in recognition ability of traditional detection models at night.

[0049] The target detection optimization method based on two-dimensional knowledge distillation proposed in this invention generates specialized model parameters suitable for nighttime and daytime scenes by employing a dual knowledge distillation mechanism combining offline and online methods on a mixed-light image set. These parameters are then integrated into a student model to construct the final target detection model. During the detection phase, the corresponding specialized model parameters are dynamically switched and loaded according to the brightness characteristics of the image to be tested, thereby enabling the model to adapt to different lighting environments. Thus, through scene-specific refined optimization, the robustness and accuracy of the target model under different lighting conditions during the day and night are effectively improved, alleviating the performance degradation problem caused by lighting differences. This achieves stable and efficient target detection capabilities in all weather conditions, solving the problems of limited adaptability of knowledge distillation methods in related technologies to different lighting and environments, as well as poor detection accuracy and stability.

[0050] Marine debris identification is a crucial task in environmental protection, especially in the automated detection of marine pollution. Due to the unique characteristics of the marine environment, marine debris detection models face numerous challenges. For example, floating debris is often affected by seawater, weather, and lighting, causing its features in images to become blurred or missing. This domain shift problem caused by environmental changes severely impacts the stability and accuracy of marine debris detection systems across different background domains. The following embodiments of the present invention will detail the target detection optimization method based on two-dimensional knowledge distillation, using a specific scenario of debris identification in a marine environment as an example. This includes the following aspects: raw data training and offline knowledge distillation, illumination domain segmentation, nighttime online knowledge distillation, daytime online knowledge distillation, multi-round optimization parameter fusion, and cross-domain switching. Specifically, a teacher model is obtained through pre-training, and then a student model with preliminary recognition capabilities is obtained through offline knowledge distillation. The student model parameters are then dynamically adjusted through an online distillation process based on illumination perception, ultimately resulting in a target detection model that can dynamically adapt to different lighting scenarios. The details are as follows: Step 11: Input the original marine debris dataset Trash-ICRA19 into the system. This dataset contains mixed images without day / night labels. Use this dataset to train a high-performing teacher model. .

[0051] Step 12: Offline Knowledge Distillation like Figure 2 As shown, this embodiment of the invention utilizes a teacher model on the Trash-ICRA19 dataset. For student models Initial distillation training is performed to generate a student model with basic recognition capabilities. A temperature-weighted softmax distillation strategy is adopted, and the loss function is as follows: ,in, KL represents the cross-entropy loss between the student model's prediction and the true label, KL represents the difference in prediction distribution between the teacher and student models, and T represents the temperature coefficient.

[0052] Step 13: Image illumination domain division like Figure 2As shown, in this embodiment of the invention, a CNN (Convolutional Neural Network) classification model is first trained using the Fomo dataset to identify the season based on the image capture time, resulting in the first CNN classification model. Then, the BDD100K dataset is used to train another CNN model, enabling it to distinguish the specific time period of the day, resulting in the second CNN classification model. The Trash-ICRA19 dataset is processed by the two CNN models and divided into a daytime image subset A and a nighttime image subset B, which serve as the input basis for subsequent online knowledge distillation.

[0053] Step 14: Nighttime Online Knowledge Distillation like Figure 3 As shown, in this embodiment of the invention, nighttime images from subset B are input sequentially. For each image processed, predictions are made by the teacher model and student model deployed in the cloud, and the distillation loss is calculated.

[0054] Write this loss into the buffer until the buffer is full of N images. Then, average the losses across all buffers to obtain:

[0055] The average loss is used for backpropagation to optimize the student model and update the weights. Each iteration runs 50 times, and the updated parameters are... Save to the cloud. Clear the buffer after completion.

[0056] Step 15: Online Knowledge Distillation During the Day Refer to step 14, as follows Figure 3 As shown, a subset of images A is input into the cloud-based teacher and student models. The prediction results are then subjected to distillation loss calculation, caching, averaging, and optimization. This process is repeated 50 times to generate a parameter set. And store it in the cloud.

[0057] Step 16: Alternate between nighttime and daytime distillation like Figure 4 As shown, in this embodiment of the invention, steps 14 and 15 are executed alternately in sequence for multiple rounds (6000 rounds) of online distillation. After each round of training, the model weights for nighttime and daytime are recorded respectively, and a parameter set is constructed: Nighttime parameter set: { } Daytime parameter set: { } Step 107: Optimal Parameter Fusion like Figure 4 As shown, after completing all rounds of training, the arithmetic mean of the parameter sets for daytime and nighttime is calculated separately to obtain the optimal parameters: ,

[0058] This step completes the optimization of the all-weather detection model and the fusion of domain parameters.

[0059] Step 108: Deployment of Real-time Illumination Sensing and Domain Switching During the actual deployment phase, after inputting the target image, the system analyzes indicators such as the image brightness histogram, HSV brightness channel mean, and exposure level to determine its lighting conditions. If the current image is a nighttime scene, then load the nighttime student model parameters. If it is a daytime scene, then load the daytime student model parameters. This enables the model to adapt to target detection during day and night cycles.

[0060] In summary, the embodiments of the present invention address the problem of significant impact from diurnal light variations in visual perception tasks such as marine debris identification, and have the following advantages: Improved robustness of detection under different lighting conditions: By combining offline distillation and online light-sensing distillation mechanisms, the model's ability to stably identify targets in daytime and nighttime images is significantly enhanced, solving the problem of reduced recognition ability of traditional detection models at night.

[0061] Avoiding error propagation caused by pseudo-labels: This invention does not rely on self-training or pseudo-label mechanisms for cross-domain optimization, thus avoiding error accumulation caused by pseudo-labels of low-quality images and improving the stability and upper limit of training accuracy.

[0062] Dynamically adapting to different lighting scenarios: The proposed lighting perception strategy can perceive image brightness features in real time and automatically load the optimal parameters (daytime parameters or nighttime parameters) during the inference stage, thereby enabling the target detection model to switch in real time and perform high-performance applications under all weather conditions.

[0063] Improving overall detection accuracy and consistency: Validation was performed on the Trash-ICRA19 dataset. The proposed algorithm significantly outperforms traditional image enhancement, self-training, and adversarial domain adaptation methods in terms of accuracy (mAP) and detection consistency. Especially in nighttime scenes, the detection accuracy is improved by an average of approximately 15%, as shown in Table 1.

[0064]

[0065] It should be noted that the target detection optimization method based on two-dimensional knowledge distillation in this embodiment of the invention can be integrated with existing mainstream target detection frameworks such as Faster R-CNN and Cascade R-CNN, and is applicable to a variety of practical application scenarios such as marine debris identification, monitoring systems, and all-weather sensing platforms. It has low engineering implementation cost and great potential for promotion.

[0066] Next, with reference to the accompanying drawings, the target detection optimization device based on two-dimensional knowledge distillation improvement proposed according to an embodiment of the present invention is described.

[0067] Figure 5 This is a block diagram of a target detection optimization device based on two-dimensional knowledge distillation according to an embodiment of the present invention.

[0068] like Figure 5 As shown, the target detection optimization device 10 based on two-dimensional knowledge distillation includes: an acquisition module 100, a training module 200, a generation module 300, and a detection module 400.

[0069] The module 100 is used to acquire a mixed-light image set; the training module 200 is used to train a pre-built teacher model using the mixed-light image set to obtain training results, and to guide a pre-built student model to perform offline knowledge distillation based on the training results; the generation module 300 is used to perform online knowledge distillation on the mixed-light image set using the trained teacher model and the offline distilled student model to generate a target detection model and acquire the image to be detected; the detection module 400 is used to extract the brightness features of the image to be detected, load the model parameters of the target detection model according to the brightness features, and use the loaded target detection model to detect the image to be detected.

[0070] In one embodiment of the present invention, the generation module 300 is further configured to: divide the mixed illumination image into a daytime image subset and a nighttime image subset using a classification model; perform nighttime online knowledge distillation on the nighttime image subset based on a pre-trained teacher model and an offline distilled student model to accumulate nighttime optimized parameter sets, and perform daytime online knowledge distillation on the daytime image subset to obtain daytime optimized parameter sets; fuse the nighttime optimized parameter sets to generate first target model parameters suitable for nighttime scenes, fuse the daytime optimized parameter sets to generate second target model parameters suitable for daytime scenes; and integrate the first target model parameters and the second target model parameters into the student model to obtain a target detection model.

[0071] In one embodiment of the present invention, the brightness features include at least one of the image grayscale mean, brightness channel mean, brightness histogram distribution, and exposure level. The detection module 400 is further used to determine the actual scene of the input image based on the brightness features. If the actual scene is a night scene, the first target model parameters are loaded to detect the image to be detected. If the actual scene is a daytime scene, the second target model parameters are loaded to detect the image to be detected.

[0072] In one embodiment of the present invention, the generation module 300 is further configured to sequentially input the mixed illumination image set into a first classification model and a second classification model, wherein the first classification model outputs seasonal data for each image in the mixed illumination image set, and the second classification model outputs temporal data for each image in the mixed illumination image set; the mixed illumination image set is divided into a daytime image subset and a nighttime image subset based on the seasonal data and temporal data.

[0073] In one embodiment of the present invention, offline knowledge distillation employs a temperature-weighted soft-label distillation strategy to minimize the total loss function of the pre-built student model during the offline knowledge distillation process, thereby obtaining the trained offline distilled student model.

[0074] In one embodiment of the present invention, the generation module 300 is further configured to acquire each night image in the night image subset; use the teacher model and the offline distilled student model to predict and calculate the distillation loss value corresponding to each night image in turn, and store it in a pre-built buffer until the buffer meets the preset conditions, and calculate the average loss value of all distillation loss values; use the average loss value to backpropagate and update the student model parameters to obtain the night optimization parameters.

[0075] In one embodiment of the present invention, the generation module 300 is further configured to acquire each daytime image in the daytime image subset; use the teacher model and the offline distilled student model to predict and calculate the distillation loss value corresponding to each daytime image in turn, and store it in a pre-built cache area until the cache area meets the preset conditions, and calculate the average loss value of all distillation loss values; use the average loss value to backpropagate and update the student model parameters to obtain the daytime optimization parameters.

[0076] It should be noted that the foregoing explanation of the target detection optimization method based on two-dimensional knowledge distillation also applies to the target detection optimization device 10 based on two-dimensional knowledge distillation in this embodiment, and will not be repeated here.

[0077] The target detection optimization device 10 based on two-dimensional knowledge distillation proposed in this embodiment of the invention generates dedicated model parameters suitable for nighttime and daytime scenes by employing a dual knowledge distillation mechanism combining offline and online methods on a mixed illumination image set. These parameters are then integrated into a student model to construct the final target detection model. During the detection phase, the corresponding dedicated model parameters are dynamically switched and loaded according to the brightness characteristics of the image to be tested, thereby enabling the model to adapt to different illumination environments. Thus, through scene-specific refined optimization, the robustness and accuracy of the target model under different daytime and nighttime illumination conditions are effectively improved, alleviating the performance degradation problem caused by illumination differences. This achieves stable and efficient target detection capabilities in all weather conditions, solving the problems of limited adaptability of knowledge distillation methods in related technologies to different illuminations and environments, as well as poor detection accuracy and stability.

[0078] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. The electronic device may include: The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0079] When the processor 602 executes the program, it implements the target detection optimization method based on two-dimensional knowledge distillation provided in the above embodiments.

[0080] Furthermore, electronic devices also include: Communication interface 603 is used for communication between memory 601 and processor 602.

[0081] The memory 601 is used to store computer programs that can run on the processor 602.

[0082] The memory 601 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0083] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0084] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0085] Processor 602 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention.

[0086] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described target detection optimization method based on two-dimensional knowledge distillation.

[0087] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0088] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0089] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0090] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0091] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0092] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A target detection optimization method based on two-dimensional knowledge distillation, characterized in that, Includes the following steps: Obtain a set of images with mixed lighting; The pre-built teacher model is trained using the mixed lighting image set to obtain training results, and the training results are used to guide the pre-built student model to perform offline knowledge distillation. The trained teacher model and the offline distilled student model are used to perform online knowledge distillation on the mixed illumination image set to generate a target detection model and obtain the image to be detected; The brightness features of the image to be detected are extracted, the model parameters of the target detection model are loaded according to the brightness features, and the loaded target detection model is used to detect the image to be detected.

2. The target detection optimization method based on two-dimensional knowledge distillation as described in claim 1, characterized in that, The step of generating a target detection model by performing online knowledge distillation on the mixed illumination image set using a trained teacher model and an offline distilled student model includes: The mixed lighting image set was divided into a daytime image subset and a nighttime image subset using a classification model; Based on the pre-trained teacher model and the offline distilled student model, the nighttime image subset is accumulated through nighttime online knowledge distillation to obtain a nighttime optimization parameter set, and the daytime image subset is accumulated through daytime online knowledge distillation to obtain a daytime optimization parameter set; The nighttime optimized parameter set is fused to generate a first target model parameter set suitable for nighttime scenarios, and the daytime optimized parameter set is fused to generate a second target model parameter set suitable for daytime scenarios; The target detection model is obtained by integrating the first target model parameters and the second target model parameters into the offline distilled student model.

3. The target detection optimization method based on two-dimensional knowledge distillation as described in claim 2, characterized in that, The brightness features include at least one of the following: image grayscale mean, brightness channel mean, brightness histogram distribution, and exposure level. The model parameters of the target detection model are loaded based on the brightness features, and the loaded target detection model is used to detect the image to be detected, including: The actual scene of the image to be detected is determined based on the brightness characteristics; If the actual scene is a nighttime scene, then the first target model parameters are loaded to detect the image to be detected; If the actual scene is a daytime scene, then the second target model parameters are loaded to detect the image to be detected.

4. The target detection optimization method based on two-dimensional knowledge distillation as described in claim 2, characterized in that, The process of using a classification model to divide the mixed lighting image into a daytime image subset and a nighttime image subset includes: The mixed-light image set is sequentially input into a first classification model and a second classification model, wherein the first classification model outputs seasonal data for each image in the mixed-light image set, and the second classification model outputs temporal data for each image in the mixed-light image set; Based on the seasonal data and the time data, the mixed illumination image set is divided into a daytime image subset and a nighttime image subset.

5. The target detection optimization method based on two-dimensional knowledge distillation as described in claim 1, characterized in that, The offline knowledge distillation employs a temperature-weighted soft-label distillation strategy to minimize the total loss function of the pre-built student model during the offline knowledge distillation process, thereby obtaining the offline-distilled student model.

6. The target detection optimization method based on two-dimensional knowledge distillation as described in claim 2, characterized in that, The pre-trained teacher model and the offline-distilled student model perform nighttime online knowledge distillation on the nighttime image subset to accumulate a nighttime optimization parameter set, including: Obtain each night image from the subset of night images; The teacher model and the offline distilled student model are used to predict and calculate the distillation loss value corresponding to each night image in turn, and store it in a pre-built cache until the cache meets the preset conditions. Then, the average loss value of all distillation loss values ​​is calculated. The average loss value is used to backpropagate and update the student model parameters to obtain the nighttime optimization parameters.

7. The target detection optimization method based on two-dimensional knowledge distillation as described in claim 2, characterized in that, Based on the pre-trained teacher model and the offline distilled student model, daytime online knowledge distillation is performed on the daytime image subset to optimize the daytime parameter set, including: Obtain each daytime image from the subset of daytime images; The teacher model and the offline distilled student model are used to predict and calculate the distillation loss value corresponding to each daytime image in turn, and store it in a pre-built cache until the cache meets the preset conditions. Then, the average loss value of all distillation loss values ​​is calculated. The average loss value is used to backpropagate and update the student model parameters to obtain the daytime optimization parameters.

8. A target detection optimization device based on two-dimensional knowledge distillation, characterized in that, include: The acquisition module is used to acquire a set of mixed lighting images; The training module is used to train a pre-built teacher model using the mixed illumination image set to obtain training results, and to guide a pre-built student model to perform offline knowledge distillation based on the training results; The generation module is used to perform online knowledge distillation on the mixed illumination image set using the trained teacher model and the offline distilled student model to generate a target detection model and obtain the image to be detected; The detection module is used to extract the brightness features of the image to be detected, load the model parameters of the target detection model according to the brightness features, and use the loaded target detection model to detect the image to be detected.

9. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the target detection optimization method based on two-dimensional knowledge distillation as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed, they implement the target detection optimization method based on two-dimensional knowledge distillation as described in any one of claims 1-7.

Images