Multi-target detection method and device based on dual-module cascading
By using a dual-module cascade method of cascading defogging and detection modules, the problems of low target detection accuracy and poor real-time performance in autonomous driving systems under foggy conditions are solved, achieving lightweight and high-precision target detection in foggy conditions, which is suitable for intelligent transportation and autonomous driving.
Patent Information
- Application Number
- CN202511480450.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-16
- Publication Date
- 2026-02-06
AI Technical Summary
Existing technologies suffer from low target detection accuracy and poor real-time performance in autonomous driving systems under foggy conditions. Traditional defogging algorithms are complex and difficult to deploy, end-to-end learning methods have a large number of parameters, and mainstream target detection models have a high rate of missed detections/false detections in foggy scenarios.
A multi-target detection method based on dual-module cascade is adopted, which achieves lightweight and high-precision target detection in foggy weather by cascading dehazing and detection modules, including K-value estimation module and image reconstruction function module, as well as YOLOv11n network.
It achieves high-precision target detection in foggy conditions, improving the detection accuracy to 43.7%, meeting the real-time requirements of autonomous driving, and is suitable for intelligent transportation and autonomous driving scenarios.
Smart Images
Figure CN121482733A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of computer vision and intelligent driving, and particularly relates to a multi-target detection method and device based on double-module cascade. BACKGROUND
[0002] In recent years, the rapid development of intelligent transportation and automatic driving systems has attracted extensive research by domestic and foreign researchers. In the automatic driving system, the image degradation problem in foggy weather seriously reduces the accuracy of target detection. The prior art has the following disadvantages:
[0003] 1. The traditional dehazing algorithm (such as dark channel prior) relies on physical model parameter estimation, which is prone to over-dehazing or complex calculation; 2. The end-to-end joint learning method has large parameter quantity and optimization difficulty, and is difficult to deploy on vehicle-mounted devices; 3. The mainstream target detection model (such as YOLO series) has high miss detection / mis-detection rate in foggy scenes due to image blur and low contrast. SUMMARY
[0004] In view of the low target detection accuracy and poor real-time performance of the automatic driving system in foggy weather, the present application provides a multi-target detection method and device based on double-module cascade, which is a light-weight and high-precision foggy multi-target detection scheme. By cascading the dehazing and detection modules, the detection performance degradation problem caused by image degradation in the prior art is solved, and the real-time requirement of automatic driving is met.
[0005] According to a first aspect of the present application, a multi-target detection method based on double-module cascade is provided, the method comprising the following steps: Step S1: A vehicle-mounted camera captures a foggy scene image to be recognized; Step S2: A dehazing module performs dehazing processing on the foggy scene image to be recognized, and inputs the dehazed image into a detection module, and the detection module outputs a detection result, the detection result including a target frame of the target to be recognized, a category of the target to be recognized, and a confidence of the category; Step S3: A vehicle-mounted control unit outputs an obstacle avoidance path based on the detection result.
[0006] Preferably, in the step S2, the dehazing module performs dehazing processing on the foggy scene image to be recognized, wherein: The defogging module comprises a K value estimation module and an image reconstruction function module, the K value estimation module comprises first, second, third, fourth, fifth convolutional layers, first, second and third feature fusion layers connected in sequence; the convolution kernel size of the first and second convolutional layers is 3x3, the convolution kernel size of the third convolutional layer is 5x5, the convolution kernel size of the fourth and fifth convolutional layers is 7x7, the first feature fusion layer fuses the output of the first convolutional layer and part of the output of the second convolutional layer, the second feature fusion layer fuses the output of the first feature fusion layer, the remaining output of the second convolutional layer and the output of the third convolutional layer, and the third feature fusion layer fuses the output of the second feature fusion layer, the output of the first convolutional layer, the output of the second convolutional layer, the output of the third convolutional layer and the output of the fourth convolutional layer; the output K(x) of the third feature fusion layer is input into the image reconstruction function module; The image reconstruction function module acquires the output K(x) of the third feature fusion layer and the to-be-identified foggy scene image, and uses a defogging image reconstruction formula to perform defogging processing, wherein, is a defogging image, is a to-be-identified foggy scene image, and b is a model constant bias, which is 1 by default.
[0007] Preferably, the detection module comprises an image processing sub-module and a YOLOv11n network, the YOLOv11n network comprises a backbone network, a neck network and a head network; the image processing sub-module performs image adjustment, normalization and format conversion on the defogging image to obtain a processed defogging image; the YOLOv11n network extracts multi-scale feature maps from the processed defogging image, inputs the multi-scale feature maps into the neck network, the neck network performs cross-scale connection and weighted fusion on the multi-scale feature maps, and inputs the generated enhanced multi-scale feature maps into the head network; the head network processes the enhanced multi-scale feature maps to obtain a detection result of the to-be-identified target.
[0008] Preferably, the foggy scene of the foggy scene image captured by the vehicle-mounted camera comprises a road, a vehicle and a pedestrian.
[0009] Preferably, the detection module comprises a C3K2 module, and the C3K2 module adopts alternating convolution kernels with scales of 3x3 and 5x5.
[0010] According to the second aspect of the present application, a multi-target detection device based on double-module cascade is provided, the device comprising: An image acquisition module configured to capture a to-be-identified foggy scene image by a vehicle-mounted camera; The recognition module is configured to perform defogging processing on the foggy scene image to be recognized by the defogging module, input the defogged image into the detection module, and output a detection result by the detection module, wherein the detection result includes a target frame of the target to be recognized, a category of the target to be recognized, and a confidence of the category. The path generation module is configured to output an obstacle avoidance path based on the detection result by the vehicle-mounted control unit.
[0011] According to a third aspect of the present application, an electronic device is provided, comprising: a processor configured to execute a plurality of instructions; a memory configured to store the plurality of instructions; wherein the plurality of instructions are stored in the memory and loaded and executed by the processor to perform the method as described above.
[0012] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the storage medium stores a plurality of instructions; the plurality of instructions are loaded and executed by a processor to perform the method as described above.
[0013] The present application can start from the original complex foggy scene, input the original picture into the defogging module, output the defogged picture, and then perform target detection, thereby integrating the defogging and detection through the cascade of the two modules.
[0014] The present application has the following beneficial technical effects: The present application can start from the original complex foggy scene, input the original picture into the defogging module, output the defogged picture, and then perform target detection, thereby integrating the defogging and detection through the cascade of the two modules, and achieving an accuracy of 43.7% in the SODA10M dataset, which is better than other YOLO series detection methods.
[0015] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, and to implement the content of the description, the following will describe the preferred embodiments of the present application in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS
[0016] The accompanying drawings, which form a part of the present application, are intended to provide a further understanding of the present application, and the present application is described as follows with reference to the accompanying drawings. In the drawings:
[0017] Figure 1 The flowchart of the multi-target detection method based on the cascade of the two modules according to an embodiment of the present application.
[0018] Figure 2 The architecture diagram of the model involved in the multi-target detection based on the cascade of the two modules according to an embodiment of the present application.
[0019] Figure 3 A detection result schematic diagram of a multi-target detection method based on double-module cascade for an embodiment of the present application. DETAILED DESCRIPTION
[0020] Firstly, a multi-target detection method based on double-module cascade for an embodiment of the present application is described. Figure 1 As shown in the figure, the method comprises the following steps: Figures 1-2
[0021] Step S1: A vehicle-mounted camera captures a foggy scene image to be recognized; Step S2: A dehazing module performs dehazing processing on the foggy scene image to be recognized, and inputs a dehazed image into a detection module, and the detection module outputs a detection result, which comprises a target frame of a target to be recognized, a category of the target to be recognized and a confidence degree of the category; Step S3: A vehicle-mounted control unit outputs an obstacle avoidance path based on the detection result.
[0022] Further, in the step S2, the dehazing module performs dehazing processing on the foggy scene image to be recognized, wherein: The dehazing module comprises a K value estimation module and an image reconstruction function module, the K value estimation module comprises first, second, third, fourth, fifth convolutional layers, first, second and third feature fusion layers connected in sequence, the convolutional kernel size of the first and second convolutional layers is 3x3, the convolutional kernel size of the third convolutional layer is 5x5, the convolutional kernel size of the fourth and fifth convolutional layers is 7x7, the first feature fusion layer fuses the output of the first convolutional layer and part of the output of the second convolutional layer, the second feature fusion layer fuses the output of the first feature fusion layer, the remaining output of the second convolutional layer and the output of the third convolutional layer, and the third feature fusion layer fuses the output of the second feature fusion layer, the output of the first convolutional layer, the output of the second convolutional layer, the output of the third convolutional layer and the output of the fourth convolutional layer; the output K(x) of the third feature fusion layer is input into the image reconstruction function module.
[0023] The image reconstruction function module acquires the output K(x) of the third feature fusion layer and the foggy scene image to be recognized, and uses a dehazed image reconstruction formula to perform dehazing processing, wherein, is the dehazed image, is the foggy scene image to be recognized, and b is a model constant bias, which is 1 by default.
[0024] Further, the detection module comprises an image processing submodule and a YOLOv11n network, the YOLOv11n network comprising a backbone network, a neck network and a head network; the image processing submodule performs image adjustment, normalization and format conversion on the defogging image to obtain a processed defogging image; the YOLOv11n network extracts a multi-scale feature map from the processed defogging image, inputs the multi-scale feature map into the neck network, and the neck network performs cross-scale connection and weighted fusion on the multi-scale feature map, and inputs the generated enhanced multi-scale feature map into the head network; the head network processes the enhanced multi-scale feature map to obtain a detection result of the to-be-recognized target.
[0025] In the present application, the foggy scene image captured by the vehicle-mounted camera includes roads, vehicles, pedestrians and the like, and a high-quality fog-free image is generated through the K value estimation module and the image reconstruction function module, the defogging image is input into the lightweight YOLOv11n network, and the target category, position and confidence are output, the detection result is directly output based on the double-head prediction mechanism, and no NMS post-processing is required.
[0026] In the present application, the defogging image is input into the detection network YOLOv11n, first, the image is adjusted, normalized and format converted to the required through input preprocessing. After processing, the feature extraction is performed through the backbone network, which includes C3K2 module, PSA attention mechanism and the like, the multi-scale feature map is output to the neck network, the features of different layers are fused through cross-scale jump connection and dynamic weighted fusion, and dynamic weights are assigned to the feature maps of different scales to generate enhanced multi-scale feature maps for prediction of the head network. The head network comprises a double-head prediction mechanism and an NMS-free output, and finally outputs a visual result containing a bounding box and a label.
[0027] Through the cascade of the two modules, an AI driving perception system based on target detection meeting the actual requirements can be obtained, which can meet the requirements of accuracy and real-time in foggy scenes.
[0028] The present application starts from a complex foggy road scene, removes the fog interference of the input foggy image in real time through the defogging module, and realizes high-precision detection in combination with the lightweight YOLOv11n network, so as to further enable the vehicle to take corresponding measures. The model reaches 43.7% on the SODA10M dataset, which significantly improves the detection accuracy and real-time performance compared with the prior art, and is suitable for automatic driving, intelligent transportation and the like.
[0029] The application provides a multi-target detection system based on double-module cascade, which is used for foggy road driving detection, and comprises a vehicle-mounted camera, a defogging module, a detection module and a control unit.
[0030] The 5-layer convolutional layers of the application extract local features, capture semantic features and enhance global context awareness respectively, and the 3-layer feature fusion layers of the application fuse the features of different layers respectively and output K(x).
[0031] Further, the detection module comprises a C3K2 module, the C3K2 module adopts an alternating convolution kernel with scales of 3*3 and 5*5 and is used for expanding the perception field; the detection module adopts a dynamic weighted fusion strategy, allocates feature weights according to the target size; and the detection module uses a double-head prediction mechanism and supports NMS-free end-to-end inference.
[0032] The application provides a plurality of embodiments.
[0033] Embodiment one Please refer to Figure 2 As shown in Figure 2 The defogging module is composed of a K value estimation module and a defogging image reconstruction function module. The detection module is composed of a backbone network, a neck network and a head network. The output of the defogging network is connected with the input of the detection network.
[0034] Embodiment two The detection method is applied to a deep learning network framework of multi-target detection based on double-module cascade. The detection method comprises the following steps:
[0035] 101、The vehicle-mounted camera captures the foggy road image and inputs the image to the defogging module; 102、The input image is predicted by the K value estimation module to obtain K(x), the K(x) and the original image I(x) are output to the defogging image J(x) by the defogging image reconstruction function, and the defogging image J(x) is input to the detection module; 103、The defogging image is first generated into a multi-scale feature map by the backbone to the neck layer, the enhanced multi-scale feature map is generated by cross-scale connection and weighted fusion to the head layer, and the regression frame and the confidence are output by the double-head prediction; 104、The detection result is transmitted to the vehicle-mounted control unit to plan an obstacle avoidance path.
[0036] In the embodiment of the present application, I(x) is a foggy road image captured by the vehicle-mounted camera, I(x) is input into the K value estimation module, the depth and concentration information of the fog and haze are captured through multi-scale feature fusion, K(x) is predicted, and then a clear image J(x) is obtained through a defogging image reconstruction function.
[0037] Embodiment three The embodiment of the present application discloses a multi-target detection device based on double-module cascade, which can include: An image acquisition module configured to capture a to-be-identified foggy scene image by a vehicle-mounted camera; An identification module configured to perform defogging processing on the to-be-identified foggy scene image by a defogging module, input the defogging image into a detection module, and output a detection result, the detection result including a target frame of the to-be-identified target, a category of the to-be-identified target, and a confidence of the category; A path generation module configured to output an obstacle avoidance path based on the detection result by a vehicle-mounted control unit.
[0038] The device disclosed by the embodiment of the present application can realize defogging and then detection by cascading the defogging and detection modules, improve the detection accuracy, keep the model lightweight, and meet the actual needs of vehicle-mounted devices.
[0039] Embodiment four The embodiment of the present application discloses a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the multi-target detection method and system based on double-module cascade described in embodiment two.
[0040] The computer readable storage medium disclosed by the embodiment of the present application can realize defogging and then detection by cascading the defogging and detection modules through the multi-target detection method based on double-module cascade, improve the detection accuracy, keep the model lightweight, and meet the actual needs of vehicle-mounted devices.
[0041] Embodiment five The embodiment of the present application discloses a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the multi-target detection method and system based on double-module cascade described in embodiment two.
[0042] The computer program product disclosed by the embodiment of the present application can realize defogging and then detection by cascading the defogging and detection modules through the multi-target detection method based on double-module cascade, improve the detection accuracy, keep the model lightweight, and meet the actual needs of vehicle-mounted devices.
[0043] The embodiment of the present application further provides an electronic device, comprising: a processor, configured to execute a plurality of instructions; a memory, configured to store the plurality of instructions; The plurality of instructions are stored in the memory and loaded and executed by the processor to perform the method.
[0044] The embodiment of the present application further provides a computer readable storage medium, wherein the storage medium stores a plurality of instructions; the plurality of instructions are loaded and executed by a processor to perform the method.
[0045] It should be noted that the embodiments and features of the present application can be combined if there is no conflict.
[0046] In the several embodiments of the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner for actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.
[0047] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.
[0048] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be a physically independent unit, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of hardware plus software function unit.
[0049] The integrated unit in the form of the software function unit can be stored in a computer readable storage medium. The software function unit is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a physical machine server, or a network cloud server, etc., which needs to be installed with an Ubuntu operating system) to execute part of steps of the method described in various embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.
[0050] The above is only a preferred embodiment of the present application, and does not limit the present application in any form. Any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present application still belongs to the scope of the technical solution of the present application.
Claims
1. A multi-target detection method based on dual-module cascade, characterized in that, The methods include: Step S1: The vehicle-mounted camera captures an image of the foggy scene to be identified; Step S2: The dehazing module performs dehazing processing on the foggy scene image to be identified, inputs the dehazed image into the detection module, and outputs the detection results, which include the target bounding box of the target to be identified, the category of the target to be identified, and the confidence level of the category. Step S3: The vehicle control unit outputs an obstacle avoidance path based on the detection results.
2. The method as described in claim 1, characterized in that, In step S2, the dehazing module performs dehazing processing on the foggy scene image to be identified, wherein: The dehazing module includes a K-value estimation module and an image reconstruction function module. The K-value estimation module consists of a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, a fifth convolutional layer, a first feature fusion layer, a second feature fusion layer, and a third feature fusion layer, all connected in sequence. The kernel size of the first and second convolutional layers is 3×3, the kernel size of the third convolutional layer is 5×5, and the kernel size of the fourth and fifth convolutional layers is 7×7. The first feature fusion layer fuses the output of the first convolutional layer and part of the output of the second convolutional layer. The second feature fusion layer fuses the output of the first feature fusion layer, the remaining output of the second convolutional layer, and the output of the third convolutional layer. The third feature fusion layer fuses the output of the second feature fusion layer, the output of the first convolutional layer, the output of the second convolutional layer, the output of the third convolutional layer, and the output of the fourth convolutional layer. The output K(x) of the third feature fusion layer is then input into the image reconstruction function module. The image reconstruction function module obtains the output K(x) of the third feature fusion layer and the foggy scene image to be identified, and uses the dehazing image reconstruction formula. Defogging treatment is performed, among which, For dehazed images, The image represents the foggy scene to be identified, and b is the model constant bias, which defaults to 1.
3. The method as described in claim 2, characterized in that, The detection module includes an image processing submodule and a YOLOv11n network. The YOLOv11n network includes a backbone network, a neck network, and a head network. The image processing submodule performs image adjustment, normalization, and format conversion on the dehazed image to obtain a processed dehazed image. The YOLOv11n network extracts multi-scale feature maps from the processed dehazed image and inputs the multi-scale feature maps into the neck network. The neck network performs cross-scale connections and weighted fusion on the multi-scale feature maps and inputs the generated enhanced multi-scale feature maps into the head network. The head network processes the enhanced multi-scale feature map to obtain the detection result of the target to be identified.
4. The method as described in claim 1, characterized in that, The foggy scene captured by the vehicle-mounted camera includes roads, vehicles, and pedestrians.
5. The method as described in claim 1, characterized in that, The detection module includes the C3K2 module, which uses alternating convolutional kernels with scales of 3×3 and 5×5.
6. A multi-target detection device based on dual-module cascade, characterized in that, The device includes: Image acquisition module: configured to capture images of foggy scenes to be identified using the vehicle-mounted camera; Recognition Module: Configured as a dehazing module to dehaze the foggy scene image to be recognized. The dehazed image is input into the detection module, and the detection module outputs the detection results, which include the target bounding box of the target to be recognized, the category of the target to be recognized, and the confidence level of the category. Path generation module: Configured for the vehicle control unit to output obstacle avoidance paths based on detection results.
7. An electronic device, comprising: A processor is used to execute multiple instructions; Memory, used to store multiple instructions; The plurality of instructions are to be stored in the memory and loaded by the processor and executed as described in any one of claims 1-5.
8. A computer-readable storage medium storing a plurality of instructions; the plurality of instructions being loaded by a processor and executing the method as claimed in any one of claims 1-5.