Night road throwing object detection method and system, electronic equipment and storage medium

By combining the methods of knowledge distillation, reflectivity learning and object detection network, a detection model based on knowledge distillation is constructed, which solves the problem of road spill detection under low light conditions at night, and achieves high-precision and real-time detection effects.

CN119785281AActive Publication Date: 2025-04-08GUANGDONG CHENGTAI TRANSPORT SCI & TECH DEV +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411714577.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-27
Publication Date
2025-04-08
Estimated Expiration
2044-11-27

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify thrown objects on the road under low light conditions at night, and deep learning models are difficult to achieve real-time detection on resource-constrained devices.

Method used

Using a method combining knowledge distillation, reflectivity learning and object detection network, a detection model based on knowledge distillation is constructed, and the comprehensive loss function is optimized to improve detection performance through joint training of student network and teacher network.

Benefits of technology

High-precision detection of road sprinklers is achieved under low light conditions, improving the generalization ability of the detection model under complex light conditions, and real-time detection is achieved on resource-constrained devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119785281A_ABST
    Figure CN119785281A_ABST
Patent Text Reader

Abstract

The invention discloses a night road throwing object detection method and system, electronic equipment and a storage medium, and the detection method comprises the steps: obtaining a night road image with a label, and carrying out the preprocessing of the image, and obtaining a preprocessed image; constructing a detection model based on knowledge distillation, wherein the detection model is composed of a student network and a teacher network; inputting the preprocessed image into a student network, and carrying out reflectivity acquisition and target detection processing to obtain a detection result A; inputting the preprocessed image into a teacher network, and carrying out target detection processing to obtain a detection result B; training the student network based on the detection results A and B; and the trained student network detects the thrown object on the night road according to the preprocessed image, and outputs an image including a thrown object bounding box and a thrown object category label. According to the detection method, technologies such as knowledge distillation, reflectivity learning and a target detection network are combined, and high-precision detection of the road spilled objects under the low-light condition is achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision target detection, and in particular to a method, system, electronic device and storage medium for detecting objects spilled on roads at night. Background Art

[0002] Objects spilled on the road may include vehicle parts, cargo, residues from traffic accidents, and other debris. These objects can pose a threat to road safety, especially at night when lighting conditions are insufficient and the driver's vision is limited. The presence of objects spilled on the road can increase road safety risks and affect nighttime driving safety.

[0003] Traditional methods for detecting spilled objects at night mainly rely on video surveillance systems and manual inspections. These methods have obvious limitations, such as low detection efficiency, poor real-time performance, and susceptibility to environmental influences. With the development of image processing technology and deep learning algorithms, road spilled object detection technology based on computer vision has gradually become a research hotspot. However, under low light conditions at night, traditional image processing technology and detection algorithms often have difficulty in accurately identifying spilled objects on the road.

[0004] In recent years, deep learning technology has achieved remarkable results in the field of object detection, especially the application of convolutional neural networks (CNN). However, deep learning models usually require a large amount of labeled data for training, and the model complexity is high, making it difficult to achieve real-time detection on resource-constrained devices. At the same time, insufficient lighting conditions at night and poor image quality under low light conditions will also affect the detection performance of the model, resulting in reduced detection accuracy. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method, system, electronic device and storage medium for detecting road spilled objects at night, which combines knowledge distillation, reflectivity learning, target detection network and other technologies to achieve high-precision detection of road spilled objects under low light conditions.

[0006] The first object of the present invention is to provide a method for detecting spilled objects on roads at night, comprising the steps of:

[0007] Obtain an annotated nighttime road image A;

[0008] Perform low-light image processing on image A to obtain a low-light image B;

[0009] Performing contrast-limited adaptive histogram equalization processing on images A and B to obtain an enhanced nighttime road image C and an enhanced low-light image D;

[0010] Constructing a detection model based on knowledge distillation, wherein the detection model consists of a student network and a teacher network;

[0011] Input images C and D into the student network, and the student network performs reflectivity training on images C and D and then performs target detection processing to obtain a detection result A including a spilled object boundary box and a probability distribution of the spilled object category;

[0012] Input images C and D into the teacher network, and the teacher network performs target detection on images C and D to obtain detection results B including the bounding box of the spilled object and the probability distribution of the spilled object category;

[0013] Based on the detection results A and B, the student network is trained, and a comprehensive loss function consisting of reflectivity loss, knowledge distillation loss, and target detection loss weights is constructed. Through the optimization of the comprehensive loss function, the parameters of the student network are continuously adjusted. When the detection performance of the student network no longer improves or reaches the preset number of iterations, the training is stopped to obtain the trained student network.

[0014] The trained student network is used to detect spilled objects on the road at night based on the preprocessed images, and output an image containing a spilled object bounding box and a spilled object category label, where the spilled object category label is the spilled object category with the highest probability.

[0015] As a further improvement of the present invention, the student network includes a reflectivity learning module and a target detection module;

[0016] The reflectivity learning module is used to perform two Retinex decompositions on image C and image D respectively to extract and learn the reflectivity information in the images;

[0017] The target detection module is used to detect the scattered objects in the image based on image C, image D, and the image after two Retinex decompositions, and output a detection result A including a boundary box of the scattered object and a probability distribution of the category of the scattered object.

[0018] As a further improvement of the present invention, the reflectivity learning module includes a Retinex decomposition submodule and a reflectivity enhancement submodule;

[0019] The Retinex decomposition submodule is used to perform initial decomposition on image C and image D respectively;

[0020] The reflectivity enhancement submodule is used to perform secondary decomposition on the image obtained by the primary decomposition by the Retinex decomposition submodule.

[0021] As a further improvement of the present invention, the step of the reflectivity enhancement submodule performing secondary decomposition on the image obtained by the primary decomposition of the Retinex decomposition submodule comprises:

[0022] The Retinex decomposition submodule performs primary decomposition on image C and image D respectively to obtain the reflectance and illuminance of the primary decomposition of image C and the reflectance and illuminance of the primary decomposition of image D;

[0023] The reflectivity enhancement submodule exchanges the reflectivity of the image C initially decomposed with the reflectivity of the image D initially decomposed, and synthesizes the image C' and the image D' after the reflectivity exchange;

[0024] Perform secondary decomposition on the image C' and the image D' respectively, and obtain the reflectivity and illuminance of the secondary decomposition of the image C' and the reflectivity and illuminance of the secondary decomposition of the image D';

[0025] The difference between the reflectance obtained by the primary decomposition and the reflectance obtained by the secondary decomposition is constrained to improve the consistency of the image reflectance.

[0026] As a further improvement of the present invention, the model adopted by the target detection module is a lightweight LeYOLO network.

[0027] As a further improvement of the present invention, the model adopted by the teacher network is a YOLOv8 network pre-trained with road images under sufficient lighting conditions.

[0028] As a further improvement of the present invention, the calculation formula of the reflectivity loss is:

[0029]

[0030] in, is the reflectivity loss, is the reflectivity of the initial decomposition of image C, is the reflectivity of the initial decomposition of image D, is the reflectivity of the secondary decomposition of image C, is the reflectivity of the secondary decomposition of image D;

[0031] The calculation formula of the knowledge distillation loss is:

[0032]

[0033] in, is the knowledge distillation loss, α is the weight parameter, For students network loss, for the loss of teacher networks;

[0034] The calculation formula of the target detection loss is:

[0035]

[0036] in, is the target detection loss, is the classification loss, is the regression loss, λ is the weight parameter for adjusting the weight between classification loss and regression loss;

[0037] The calculation formula of the comprehensive loss function is:

[0038]

[0039] in, is a comprehensive loss function, and β1, β2, and β3 are weight parameters of each loss.

[0040] The second object of the present invention is to provide a nighttime road spill detection system, comprising:

[0041] The image preprocessing module is used to obtain an annotated nighttime road image A; perform low-light image processing on image A to obtain a low-light image B; perform contrast-limited adaptive histogram equalization processing on image A and image B to obtain an enhanced nighttime road image C and an enhanced low-light image D;

[0042] A detection model building module, used to build a detection model based on knowledge distillation, wherein the detection model consists of a student network and a teacher network;

[0043] The detection model training module is used to input images C and D into the student network, and the student network performs reflectivity training on images C and D and then performs target detection processing to obtain a detection result A including a spilled object bounding box and a probability distribution of the spilled object category; input images C and D into the teacher network, and the teacher network performs target detection processing on images C and D to obtain a detection result B including a spilled object bounding box and a probability distribution of the spilled object category; based on the detection results A and B, the student network is trained to construct a comprehensive loss function composed of reflectivity loss, knowledge distillation loss, and target detection loss weights, and the parameters of the student network are continuously adjusted through the optimization of the comprehensive loss function. When the detection performance of the student network no longer improves or reaches a preset number of iterations, the training is stopped to obtain a trained student network;

[0044] The spilled object detection module is based on the trained student network and is used to detect spilled objects on the road at night according to the preprocessed images, and output an image containing the spilled object boundary box and the spilled object category label, where the spilled object category label is the spilled object category with the highest probability.

[0045] The third object of the present invention is to provide an electronic device, comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above-mentioned detection method.

[0046] A fourth object of the present invention is to provide a storage medium storing a computer program executable by an electronic device, wherein when the program runs on the electronic device, the electronic device executes the above-mentioned detection method.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] A detection model based on knowledge distillation is constructed. The detection results of the teacher network in the detection model are passed to the student network using knowledge distillation, so that the student network can learn the feature extraction and classification capabilities of the teacher network, thereby enhancing the generalization ability of the student network under complex lighting conditions, thereby improving the high-precision detection capability of the student network.

[0049] By preprocessing night road images, the contrast and clarity of night road images are effectively improved, the interference of noise and shadows is reduced, and richer visual information is provided for subsequent spilled object detection. This enables the detection model to effectively cope with complex lighting conditions in low-light environments and improves the detection capability of the detection model in low-light environments.

[0050] Based on the reflectivity learning method of Retinex theory, the reflectivity learning module is introduced into the student network, and the reflectivity decomposition and reflectivity enhancement techniques are combined to effectively improve the contrast and details of the image, making the spilled objects more clearly identifiable in the image, thereby significantly improving the accuracy of detection. At the same time, the student network adopts a lightweight LeYOLO network structure, which reduces the computational complexity of the model while achieving rapid detection of spilled objects, so as to achieve efficient spilled object detection on embedded devices and mobile devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flow chart of a method for detecting spilled objects on roads at night;

[0052] Figure 2 This is a schematic diagram of the overall architecture of the nighttime road spill detection model;

[0053] Figure 3 Schematic diagram of the process of training the student network;

[0054] Figure 4 A schematic diagram of the process of processing the pre-processed image by the reflectivity learning module;

[0055] Figure 5 This is a schematic diagram of the structure of the nighttime road spilled object detection system. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0058] See also Figure 1 This embodiment provides a method for detecting spilled objects on a road at night, comprising the steps of:

[0059] Obtain an annotated nighttime road image A; perform low-light image (Dark ISP) processing on image A to obtain a low-light image B simulating a more extreme low-light condition; perform contrast-limited adaptive histogram equalization (CLAHE) processing on image A and image B to obtain an enhanced nighttime road image C and an enhanced low-light image D;

[0060] In order to cope with the challenges of low-light environment for detecting spilled objects on nighttime highways, the nighttime images input to the detection model are preprocessed to generate input data that is more suitable for the detection task. The goal of this preprocessing process is to enhance the contrast and details of the image, thereby improving the model's ability to detect objects in low-light environments.

[0061] Dark ISP adjusts the light intensity of the image so that the generated low-light image can more realistically reflect the visual characteristics of the low-light environment. This processing step provides the detection model with input under a variety of lighting conditions to enhance the detection model's lighting adaptability.

[0062] CLAHE technology significantly improves the details in the image by enhancing the local contrast of the image, especially when the brightness difference between the scattered objects and the background is small, it can make these objects more prominent. By performing local histogram equalization on the image, the noise amplification problem that may be caused by traditional global equalization is avoided, while maintaining the naturalness of the image.

[0063] After each annotated night road image A is processed by Dark ISP and CLAHE, a set of enhanced images (images C and D) are obtained. These images are used as input data of the detection model, providing richer visual information for subsequent feature extraction and target detection, enabling the detection model to effectively cope with complex lighting conditions in low-light environments at night and improve the detection model's target detection capabilities in low-light environments.

[0064] Construct a detection model based on knowledge distillation, named MSLAD-Net (Multi-Source LightAdaptation Detection Network), which consists of a student network and a teacher network;

[0065] Input image C and image D into the student network, the student network includes a reflectivity learning module and a target detection module; the reflectivity learning module is used to perform two Retinex decompositions on image C and image D respectively to extract and learn the reflectivity information in the image; the target detection module is used to detect the spilled objects in the image according to image C, image D and the image after the two Retinex decompositions, and output a detection result A including a spilled object boundary box and a probability distribution of the spilled object category;

[0066] Input images C and D into a teacher network. The model used by the teacher network is a YOLOv8 network pre-trained with road images under sufficient lighting conditions. The teacher network performs high-precision target detection processing on images C and D to obtain a detection result B including a spilled object boundary box and a probability distribution of the spilled object category.

[0067] Based on the detection results A and B, the student network is trained, and a comprehensive loss function consisting of reflectivity loss, knowledge distillation loss, and target detection loss weights is constructed. Through the optimization of the comprehensive loss function, the parameters of the student network are continuously adjusted. When the detection performance of the student network no longer improves or reaches the preset number of iterations, the training is stopped to obtain the trained student network.

[0068] The trained student network is used to detect spilled objects on the road at night based on the preprocessed images, and output an image containing the spilled object bounding box and the spilled object category label, where the spilled object category label is the spilled object category with the highest probability.

[0069] The detection method combines knowledge distillation, reflectivity learning, target detection network and other technologies to achieve high-precision detection of road spills under low-light conditions. The overall architecture of the detection model MSLAD-Net and its detection process are as follows: Figure 2 shown.

[0070] Specifically, in order to effectively cope with the complex lighting conditions in low-light environments at night, a reflectance learning module is introduced into the student network based on the reflectance learning method based on Retinex theory. The reflectance learning module performs primary Retinex decomposition on image C and image D respectively to extract the reflectance information in the images, and performs secondary decomposition based on the primary decomposition to further learn the changes in reflectance under different lighting conditions.

[0071] Retinex theory assumes that images can be decomposed into two parts: reflectance and illumination. The reflectance part is not affected by illumination changes and retains the inherent characteristics of the scene. This feature makes reflectance learning important when dealing with illumination changes, especially in low-light environments at night, where it can enhance the detection model's ability to recognize target objects.

[0072] Specifically, the reflectivity learning module includes a Retinex decomposition submodule and a reflectivity enhancement submodule; the reflectivity learning module processes image C and image D respectively as follows: Figure 4 As shown;

[0073] The Retinex decomposition submodules respectively decompose the image C(I n ), image D(I l ) is first decomposed to obtain image C(I n )Reflectivity of the initial decomposition Illumination and image D(I l )Reflectivity of the initial decomposition Illumination

[0074] The reflectivity enhancement submodule converts the image C(I n )Reflectivity of the initial decomposition and image D(I l )Reflectivity of the initial decomposition Perform the exchange, and synthesize the image C' and the image D' after the reflectivity exchange;

[0075] For image C'(I' n ) and image D'(I l ') is decomposed twice to obtain image C'(I' n )Reflectivity of secondary decomposition Illumination and image D'(I l ') Reflectivity of secondary decomposition Illumination

[0076] The difference between the reflectivity obtained from the primary decomposition and the reflectivity obtained from the secondary decomposition Constraints are imposed to improve the consistency of image reflectivity.

[0077] The reflectivity learning module combines reflectivity decomposition and reflectivity enhancement technology to effectively improve the contrast and details of the image, making the spilled objects more clearly visible in the image, thereby significantly improving the accuracy of detection.

[0078] Furthermore, in order to achieve efficient target detection in a resource-constrained environment, the model used by the target detection module is a lightweight LeYOLO network. The lightweight LeYOLO network includes an inverted bottleneck module, a fast pyramid architecture network, an efficient backbone network, and a decoupled network head; the inverted bottleneck module forms the basis of the efficient backbone network, and the features extracted by the efficient backbone network are multi-scale fused through the fast pyramid architecture network, and finally the decoupled network head outputs an image containing the boundary box of the scattered object and the category label of the scattered object.

[0079] LeYOLO is designed to achieve real-time and efficient target detection capabilities on embedded devices and edge computing platforms by minimizing computational costs (FLOPs). The core design concept of LeYOLO is to optimize computational efficiency based on lightweight convolution operations and efficient multi-scale feature sharing. The inverted bottleneck module (Inverted Bottleneck) and the fast pyramid architecture network (FPANet) are introduced into its architecture. These modules can significantly reduce the computational complexity of the network while maintaining high detection accuracy.

[0080] Inverted Bottleneck: This module was originally proposed by MobileNetV2 and is widely used in various lightweight networks. It reduces the amount of computation by reducing the number of channels and using point-by-point convolutions, while increasing the depth of the intermediate feature map by the expansion factor to capture more spatial information. In LeYOLO, the inverted bottleneck module is optimized for more efficient channel expansion, ensuring that the computational requirements on large feature maps are minimized.

[0081] Fast Pyramid Architecture Network (FPANet): LeYOLO also designed a fast pyramid architecture network for efficient sharing of feature information between different semantic levels. FPANet achieves fast sharing of multi-scale features by reducing the number of convolutional layers and channels, thereby further reducing computational overhead.

[0082] Efficient Backbone Feature Extractor: LeYOLO uses an efficient backbone network, using the inverted bottleneck module as the basic building block, combined with the information bottleneck characteristic, to minimize the information exchange between layers, thereby reducing the overall computational load. The channel expansion strategy guided by the information bottleneck theory can ensure that the number of input / output channels at each level is maintained within a reasonable range to achieve the best information transmission efficiency.

[0083] Decoupled Network Head (DNiN Head): The decoupled network head in LeYOLO separates the classification and bounding box regression tasks, and uses independent pointwise convolutions to process them. This design not only simplifies the calculation process, but also further refines the spatial relationship through lightweight depthwise convolutions, thereby improving detection accuracy.

[0084] In the MSLAD-Net detection model, LeYOLO, as the core part of the student network, provides a reliable foundation for the reflectivity learning module and the target detection module through its efficient feature extraction capabilities. LeYOLO's miniaturized architecture ensures that the model can adapt to the resource constraints of embedded devices and edge computing platforms while maintaining high-precision detection. By combining LeYOLO, the MSLAD-Net detection model can not only achieve robust target detection in low-light environments, but also maintain excellent computational efficiency.

[0085] In order to improve the detection performance of the MSLAD-Net detection model in low-light environments at night, the knowledge distillation technology was introduced to transfer the knowledge of the pre-trained teacher network (YOLOv8) to the student network, thereby enhancing the generalization ability of the student network under complex lighting conditions.

[0086] The core idea of ​​knowledge distillation is to use a superior and complex teacher network to provide soft labels as additional supervision signals during the student network training process, thereby improving the learning effect of the student network. In the design of the MSLAD-Net detection model, the model used by the teacher network is a YOLOv8 network pre-trained with road images under sufficient lighting conditions, which enables the teacher network to generate high-precision target detection results.

[0087] During the training process of the student network, the student network is not only trained by directly learning the labels (HardLabels) of the annotated data, but also by learning the soft labels generated by the teacher network to refine its feature expression ability. Specifically, the detection results output by the teacher network (images with soft labels) contain more information about the probability distribution of target classification, and the student network can better capture the subtle features of the target when learning these probability distributions, especially in low-light environments. The process of training the student network is as follows: Figure 3 shown.

[0088] In order to ensure that the detection model can efficiently detect targets in low-light environments, a loss function that integrates reflectance learning, knowledge distillation, and multiple requirements of target detection is designed to ensure that the output of the detection model can maintain high accuracy and robustness under different lighting conditions.

[0089] During the training process, reflectance loss is used to ensure the accuracy of reflectance learning and the consistency of image reflectance under different lighting conditions. By constraining the difference between the reflectance obtained by the primary decomposition and the secondary decomposition, the detection model can better learn the features that are invariant to lighting, thereby improving the detection accuracy of the detection model in low-light environments.

[0090] The reflectivity loss is defined as the difference between the reflectivities obtained after two decompositions of the preprocessed image, and is calculated as:

[0091]

[0092] in, is the reflectivity loss, is the reflectivity of the initial decomposition of image C, is the reflectivity of the initial decomposition of image D, is the reflectivity of the secondary decomposition of image C, is the reflectivity of the secondary decomposition of image D.

[0093] In the process of training the student network, the student network loss (Student Loss) and the teacher network loss (Teacher Loss) are combined for joint optimization. Through the weighted sum method, the weights of these two parts of the loss are dynamically adjusted during the training process, so as to ensure that the student network learns the knowledge of the teacher network while not losing the ability to directly learn the labeled data. In the final detection stage, the student network achieves high-precision target detection under complex lighting conditions by combining the feature expression ability obtained by knowledge distillation and the illumination adaptability in low-light environments.

[0094] The calculation formula of knowledge distillation loss is:

[0095]

[0096] in, is the knowledge distillation loss, α is the weight parameter, For students network loss, for the loss of teacher networks;

[0097] The object detection loss is the core loss of MSLAD-Net, which consists of classification loss and bounding box regression loss and is used to directly optimize the object detection results.

[0098] The calculation formula for target detection loss is:

[0099]

[0100] in, is the target detection loss, is the classification loss, is the regression loss, λ is the weight parameter for adjusting the weight between classification loss and regression loss;

[0101] The final comprehensive loss function consists of reflectivity loss, knowledge distillation loss, and object detection loss weighted to guide the training of the student network.

[0102] The calculation formula of the comprehensive loss function is:

[0103]

[0104] in, is a comprehensive loss function, β1, β2, and β3 are the weight parameters of each loss. By adjusting β1, β2, and β3, the detection model can balance the influence of each loss item during the training process, thereby achieving the optimal detection performance.

[0105] After training, the detection model finally outputs the target detection result, which is an image containing the spilled object bounding box and the spilled object category label. Thanks to the optimization of the comprehensive loss function, the detection model can maintain high detection accuracy and robustness under different lighting conditions, thus meeting the actual needs of nighttime highway spilled object detection.

[0106] Please refer to Figure 5 This embodiment provides a nighttime road spill detection system, including:

[0107] The image preprocessing module is used to obtain an annotated nighttime road image A; perform low-light image processing on image A to obtain a low-light image B; perform contrast-limited adaptive histogram equalization processing on image A and image B to obtain an enhanced nighttime road image C and an enhanced low-light image D;

[0108] A detection model building module, used to build a detection model based on knowledge distillation, wherein the detection model consists of a student network and a teacher network;

[0109] The detection model training module is used to input images C and D into the student network, and the student network performs reflectivity training on images C and D and then performs target detection processing to obtain a detection result A including a spilled object bounding box and a probability distribution of the spilled object category; input images C and D into the teacher network, and the teacher network performs target detection processing on images C and D to obtain a detection result B including a spilled object bounding box and a probability distribution of the spilled object category; based on the detection results A and B, the student network is trained to construct a comprehensive loss function composed of reflectivity loss, knowledge distillation loss, and target detection loss weights, and the parameters of the student network are continuously adjusted through the optimization of the comprehensive loss function. When the detection performance of the student network no longer improves or reaches a preset number of iterations, the training is stopped to obtain a trained student network;

[0110] The spilled object detection module is based on the trained student network and is used to detect spilled objects on the road at night according to the preprocessed images, and output an image containing the spilled object boundary box and the spilled object category label, where the spilled object category label is the spilled object category with the highest probability.

[0111] This embodiment provides an electronic device, including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the above detection method.

[0112] This embodiment provides a storage medium storing a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the above detection method.

[0113] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting spilled objects on roads at night, characterized in that: Includes steps: Obtain an annotated nighttime road image A; Perform low-light image processing on image A to obtain a low-light image B; Performing contrast-limited adaptive histogram equalization processing on images A and B to obtain an enhanced nighttime road image C and an enhanced low-light image D; Constructing a detection model based on knowledge distillation, wherein the detection model consists of a student network and a teacher network; Input images C and D into the student network, and the student network performs reflectivity training on images C and D and then performs target detection processing to obtain a detection result A including a spilled object boundary box and a probability distribution of the spilled object category; Input images C and D into the teacher network, and the teacher network performs target detection on images C and D to obtain detection results B including the bounding box of the spilled object and the probability distribution of the spilled object category; Based on the detection results A and B, the student network is trained, and a comprehensive loss function consisting of reflectivity loss, knowledge distillation loss, and target detection loss weights is constructed. Through the optimization of the comprehensive loss function, the parameters of the student network are continuously adjusted. When the detection performance of the student network no longer improves or reaches the preset number of iterations, the training is stopped to obtain the trained student network. The trained student network is used to detect spilled objects on the road at night based on the preprocessed images, and output an image containing a spilled object bounding box and a spilled object category label, where the spilled object category label is the spilled object category with the highest probability.

2. A method for detecting spilled objects on roads at night according to claim 1, characterized in that: The student network includes a reflectivity learning module and a target detection module; The reflectivity learning module is used to perform two Retinex decompositions on image C and image D respectively to extract and learn the reflectivity information in the images; The target detection module is used to detect the scattered objects in the image based on image C, image D, and the image after two Retinex decompositions, and output a detection result A including a boundary box of the scattered object and a probability distribution of the category of the scattered object.

3. A method for detecting spilled objects on roads at night according to claim 2, characterized in that: The reflectivity learning module includes a Retinex decomposition submodule and a reflectivity enhancement submodule; The Retinex decomposition submodule is used to perform initial decomposition on image C and image D respectively; The reflectivity enhancement submodule is used to perform secondary decomposition on the image obtained by the primary decomposition by the Retinex decomposition submodule.

4. A method for detecting spilled objects on roads at night according to claim 3, characterized in that: The step of the reflectivity enhancement submodule performing secondary decomposition on the image obtained by the primary decomposition of the Retinex decomposition submodule comprises: The Retinex decomposition submodule performs primary decomposition on image C and image D respectively to obtain the reflectance and illuminance of the primary decomposition of image C and the reflectance and illuminance of the primary decomposition of image D; The reflectivity enhancement submodule exchanges the reflectivity of the image C initially decomposed with the reflectivity of the image D initially decomposed, and synthesizes the image C' and the image D' after the reflectivity exchange; Perform secondary decomposition on the image C' and the image D' respectively, and obtain the reflectivity and illuminance of the secondary decomposition of the image C' and the reflectivity and illuminance of the secondary decomposition of the image D'; The difference between the reflectance obtained by the primary decomposition and the reflectance obtained by the secondary decomposition is constrained to improve the consistency of the image reflectance.

5. A method for detecting spilled objects on roads at night according to any one of claims 2 to 4, characterized in that: The model used by the target detection module is a lightweight LeYOLO network.

6. A method for detecting spilled objects on roads at night according to claim 1, characterized in that: The model used by the teacher network is a YOLOv8 network pre-trained with road images under sufficient lighting conditions.

7. A method for detecting spilled objects on roads at night according to claim 2, characterized in that: The calculation formula of the reflectivity loss is: in, is the reflectivity loss, is the reflectivity of the initial decomposition of image C, is the reflectivity of the initial decomposition of image D, is the reflectivity of the secondary decomposition of image C, is the reflectivity of the secondary decomposition of image D; The calculation formula of the knowledge distillation loss is: in, is the knowledge distillation loss, α is the weight parameter, For students network loss, for the loss of teacher networks; The calculation formula of the target detection loss is: in, is the target detection loss, is the classification loss, is the regression loss, λ is the weight parameter for adjusting the weight between classification loss and regression loss; The calculation formula of the comprehensive loss function is: in, is a comprehensive loss function, and β1, β2, and β3 are weight parameters of each loss.

8. A nighttime road spill detection system, used to implement the detection method according to any one of claims 1 to 7, characterized in that: include: An image preprocessing module, used to obtain an annotated nighttime road image A; Perform low-light image processing on image A to obtain a low-light image B; Performing contrast-limited adaptive histogram equalization processing on images A and B to obtain an enhanced nighttime road image C and an enhanced low-light image D; A detection model building module, used to build a detection model based on knowledge distillation, wherein the detection model consists of a student network and a teacher network; The detection model training module is used to input images C and D into the student network, and the student network performs reflectivity training on images C and D and then performs target detection processing to obtain a detection result A including a spilled object bounding box and a probability distribution of the spilled object category; input images C and D into the teacher network, and the teacher network performs target detection processing on images C and D to obtain a detection result B including a spilled object bounding box and a probability distribution of the spilled object category; based on the detection results A and B, the student network is trained to construct a comprehensive loss function composed of reflectivity loss, knowledge distillation loss, and target detection loss weights, and the parameters of the student network are continuously adjusted through the optimization of the comprehensive loss function. When the detection performance of the student network no longer improves or reaches a preset number of iterations, the training is stopped to obtain a trained student network; The spilled object detection module is based on the trained student network and is used to detect spilled objects on the road at night according to the preprocessed images, and output an image containing the spilled object boundary box and the spilled object category label, where the spilled object category label is the spilled object category with the highest probability.

9. An electronic device, characterized in that: The invention comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit executes the detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: It stores a computer program executable by an electronic device, and when the program runs on the electronic device, the electronic device executes the detection method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Night muck truck detection method based on knowledge distillation

    CN114973156A

  • Low-illumination image enhancement method and system and readable storage medium

    CN115170415A