A target detection method and system based on multi-branch weighted fusion

Abstract

This invention discloses a target detection method and system based on multi-branch weighted fusion. The method includes: acquiring and labeling scene images under different lighting conditions to form a multi-lighting image dataset; designing a first branch with adaptive lighting enhancement capabilities and its feature extractor, a second branch feature extractor that directly extracts features from the original image, and a fusion branch for multi-source feature fusion, and constructing a target detection model accordingly; performing end-to-end training on the model to optimize model parameters; inputting the image to be detected into the trained model, and outputting the final target detection result. The system includes a dataset construction unit, a model construction unit, a model training unit, and a detection unit. By using this invention, the accuracy of target detection in low-light scenes can be improved. This invention can be widely applied in the field of computer vision.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a target detection method and system based on multi-branch weighted fusion. Background Technology

[0002] With accelerated urbanization and scarce land resources, the construction of underground parking garages has become a common choice for modern buildings (such as commercial centers, residential buildings, and transportation hubs), and their safety and efficient management are increasingly important. Although target detection technology is becoming increasingly mature in open road scenarios, its performance in enclosed environments such as underground parking garages, especially under extremely low light conditions, still faces severe challenges.

[0003] Current research on images in low-light environments primarily focuses on image enhancement methods. These methods aim to improve the visual quality of images in low-light conditions, making them appear closer to images under normal lighting. While existing illumination enhancement algorithms can improve the visibility of low-light images, their optimization mechanisms, independent of the detection task, have fundamental flaws in the extremely low-light environment of an underground parking lot. First, indiscriminate enhancement can irreversibly damage the original scene information. For example, over-brightening leads to texture distortion and noise amplification, causing key targets to lose details or introduce artifacts during enhancement, resulting in a surge in both false negative and false positive rates. Summary of the Invention

[0004] In view of this, in order to solve the technical problem that existing target detection methods in low-light environments use illumination enhancement to make the image approximate that of normal-light images, but blind enhancement will lead to texture distortion, noise amplification, and thus low detection accuracy, the present invention proposes a target detection method based on multi-branch weighted fusion, the method comprising the following steps:

[0005] First, scene images under different lighting conditions are collected and finely annotated to form a multi-lighting image dataset;

[0006] Subsequently, a three-branch network structure is designed, including: a first branch with adaptive illumination enhancement capability and its feature extractor, a second branch feature extractor that directly extracts features from the original image, and a fusion branch for multi-source feature fusion;

[0007] The network was trained end-to-end based on the constructed dataset, and the model parameters were optimized to obtain an illumination-adaptive target detection model.

[0008] During the inference phase, the image to be detected is input into the trained model, and the final object detection result is output.

[0009] In some embodiments of the first aspect, a noise suppression attention module is provided in the fusion branch.

[0010] Secondly, based on the above method, the present invention also proposes a target detection system based on multi-branch weighted fusion, the system comprising:

[0011] The dataset construction unit is used to collect scene images under different lighting conditions and perform fine annotation;

[0012] The model building unit is used to build a target detection model with a three-branch network structure.

[0013] The model training unit uses the constructed dataset to train the membrane label detection model end-to-end;

[0014] The detection unit applies the trained model and outputs the target detection results.

[0015] Based on the above scheme, this invention provides a target detection method and system based on multi-branch weighted fusion, which effectively integrates the visibility of dark areas and the authenticity of information, overcoming the limitations of a single branch. The weighted fusion of this invention cleverly combines enhanced information and original robust information, and focuses on key regions and channels for target detection (especially low-light sensitive targets) in the fusion branches, while effectively suppressing noise or degraded background interference, thereby achieving better overall detection performance. Attached Figure Description

[0016] Figure 1 This is a flowchart of the steps of a target detection method based on multi-branch weighted fusion according to the present invention;

[0017] Figure 2 This is a schematic diagram of the framework of the target detection model in a specific embodiment of the present invention. Detailed Implementation

[0018] To address the technical problem of blindly enhancing illumination (i.e., a single branch) in the background art, which leads to low detection accuracy, this invention adds another branch that retains display information, thereby obtaining more reliable features for target detection.

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0021] It should be understood that the terms "system," "apparatus," "unit," and / or "module" used in this application are a method of distinguishing different components, elements, parts, sections, or assemblies at different levels. However, if other terms can achieve the same purpose, they may be replaced by other expressions.

[0022] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "a," and / or "the" are not specifically singular and may include the plural. Generally, the terms "comprising" and "including" only indicate the inclusion of expressly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements. An element defined by the phrase "comprising an..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0023] In the description of the embodiments of this application, "a plurality of" refers to two or more. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0024] Furthermore, flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed precisely in sequence. Instead, the steps can be processed in reverse order or simultaneously. Additionally, other operations can be added to these processes, or one or more steps can be removed from them.

[0025] Reference Figure 1 This is a flowchart illustrating an optional example of the target detection method based on multi-branch weighted fusion proposed in this invention. This method can be applied to computer devices, and the target detection method proposed in this embodiment may include, but is not limited to, the following steps:

[0026] Step S1: Obtain images under different lighting conditions and perform image annotation to construct an image dataset;

[0027] Step S2: Construct an object detection model based on the first branch, the second branch, and the fusion branch; the first branch includes an adaptive illumination enhancement module and a first feature extractor, the second branch includes a second feature extractor, and the third branch has a noise suppression attention module.

[0028] The framework of the object detection model is referenced here. Figure 2 .

[0029] Step S3: Train the object detection model based on the image dataset;

[0030] Step S4: Input the image to be tested into the trained target detection model to generate the detection result.

[0031] In some feasible embodiments, the first branch in the object detection model, also called the illumination enhancement branch, specifically includes:

[0032] The illumination enhancement branch consists of a plug-and-play adaptive illumination enhancement module and a feature extractor. The plug-and-play adaptive illumination enhancement module can be replaced with illumination enhancement network structures from existing research (e.g., ), specifically responsible for handling low-light degradation. The image after illumination enhancement is obtained through a plug-and-play adaptive illumination enhancement module. The illumination enhancement branch also includes a feature extractor. This feature extractor enhances weak features in dark areas while suppressing noise interference through progressive downsampling and cross-layer feature reuse via the C2f module. Its channel expansion strategy balances computational efficiency and feature representation capability, making it particularly suitable for extracting weak signal targets such as vehicle outlines and blurred pedestrians in low-light scenes of underground parking lots. This architecture achieves initial noise filtering through three convolutions with a stride of 2 and captures features of distant targets at an 80×80 resolution layer, adaptively handling targets at different distances within the parking lot. The enhanced features are obtained by feeding them into the feature extractor. .

[0033] In some feasible embodiments, the second branch in the object detection model, also called the original image branch, specifically includes:

[0034] The original image branch preserves real-world scene information and can be seen as a complement to the illumination enhancement branch. It mitigates noise or over-enhancement in the illuminated image. For simplicity, the feature extractor for this branch is the same as that for the illumination enhancement branch. In the original image branch, the input... Low-light robust features can then be extracted from them. It is worth noting that the parameters are not shared between the illumination enhancement branch and the original image branch, in order to facilitate the modeling of different illumination conditions from the input image.

[0035] In some feasible implementations, the fusion branch in the object detection model, also called the main branch, specifically includes:

[0036] To effectively fuse the complementary information from the two branches (the visibility of dark areas in the illumination enhancement branch and the information authenticity of the original image branch), the feature maps output by the two branches are first weighted and summed. The result of the weighted sum is as follows:

[0037]

[0038] in It is a robust feature under low light conditions. For illumination enhancement features, Weights for low-light robustness features. These are the weights for the illumination enhancement features. The two weights can be adjusted according to the random conditions of the low-light underground environment to achieve a balance between minimizing image loss during illumination enhancement and extracting more robust features.

[0039] Then, in the main branch, the weighted sum of the feature information from each branch is passed through the noise suppression attention module and input into the subsequent network structure. The overall network structure is constrained by the following loss function.

[0040]

[0041] in, It is the bounding box regression loss. It is classification loss. It is the distribution focus loss. These are hyperparameters that control the weights of each loss term, used to balance the losses of different parts.

[0042] The attention module is divided into two parallel parts. This paper assumes that the input noise suppression attention module... The first part aims to capture the relationship between channels and space simultaneously through cross-dimensional interaction (channel-height, channel-width, spatial dimension), enhancing key target features (such as headlight outlines and vehicle body reflections) and improving the model's feature discrimination ability in low light. First, let's explain the tools needed for the first part. Pooling layer. The function of the pooling layer is to reduce the channel dimension of the tensor from C to 2, which is achieved by concatenating the results of max pooling and average pooling along this dimension. The mathematical formula for the pooling layer is expressed as follows.

[0043]

[0044] in and These represent max pooling and average pooling along the 0th dimension, respectively.

[0045] The first part comprises three branches, performing CH interaction, CW interaction, and HW interaction respectively. In low-light target detection in underground parking lots, this approach effectively overcomes viewpoint distortion and maintains robustness against targets with unconventional poses. The outputs of the three branches are fused through simple averaging to obtain the final output. for:

[0046]

[0047] in For the Sigmoid function, For a k×k convolutional layer in three branches, , These are the results obtained by rotating tensor x during CH and CW interactions, respectively. They are respectively The result obtained by performing Z-pooling.

[0048] The second part aims to dynamically focus on key regions to improve feature discrimination under low light conditions. The input for the second part remains the same. First, a uniform grid of points is generated as reference points. Then, we normalize these coordinates to the range [-1, 1]. Finally, we sample features at the deformation point locations as keys and values, and process them using a projection matrix:

[0049] in and These represent the transformed key embedding and value embedding, respectively. The sampled value is obtained from x using a sampling function. Similar to existing methods, this embodiment performs multi-head attention calculation on q, k, and v, and uses a relative position offset R. In low-light target detection in underground parking lots, this method is beneficial for feature decoupling enhancement and motion robustness optimization. The output formula of a single attention head is as follows:

[0050]

[0051]

[0052] in This corresponds to the location embedding method followed in previous studies. The features of each attention head are concatenated and then... projection( The final output is obtained from the projection matrix. As shown in formula (7).

[0053] The noise suppression attention module simultaneously captures channel and spatial relationships through cross-dimensional interaction, enhancing key target features, and improves feature discrimination under low light conditions by dynamically focusing on key areas. The final output after integration is:

[0054]

[0055] In some feasible embodiments, step S1 specifically includes:

[0056] S1.1. Acquire images under different lighting conditions;

[0057] Specifically, to construct the SubDark low-light image dataset for underground scenes, this embodiment uses an iPhone 13 camera to record video clips in a typical underground parking lot environment. During the acquisition of the original video footage, we captured both well-lit parking areas and dark corners not covered by light, and simulated different image lighting changes by varying camera exposure. Therefore, the dataset has rich lighting variations, covering images from normal lighting to extremely low lighting, to simulate lighting changes that may exist in real-world scenes. After frame extraction and filtering, a total of 3438 valid image samples were obtained. Of these samples, approximately 16.5% (567 images) are images under normal lighting conditions, while the remainder cover low-light scene images with different intensities and distribution characteristics. This sample composition aims to ensure that the dataset mainly consists of low-light scenes, while also including a small number of normal lighting reference samples for algorithm comparison or robustness testing.

[0058] S1.2, Perform image enhancement on the low-light images;

[0059] Specifically, one of the challenges of this dataset is image annotation, as most of the data was acquired in extremely low lighting conditions, or even near-total darkness. The original images often exhibit very low brightness, significant noise, and blurred details, making it difficult to clearly identify key targets and severely hindering manual annotators from performing accurate bounding box drawing, semantic segmentation, or keypoint labeling. To overcome this obstacle and effectively improve image annotability, this embodiment employs the ZeroDCE++ low-light image enhancement algorithm to preprocess the original data. This method adaptively and non-linearly enhances overall brightness and contrast while effectively suppressing noise and preserving image details as much as possible. After ZeroDCE++ enhancement, image quality is significantly improved; the outlines and textures of targets previously obscured by darkness are revealed, ensuring that key targets in the image are at least clearly distinguishable to the naked eye.

[0060] S1.3. Label the images to obtain the image dataset.

[0061] Specifically, after image illumination enhancement, the research team used PPOCRLabel software to annotate the images, annotating a total of 10,623 instances across 12 categories. The number of images and instances for each category is shown in Table 1.

[0062] Table 1: Number of images and instances for each target type

[0063]

[0064] Based on the methods and data of the above embodiments, the present invention also provides related simulation examples:

[0065] This embodiment uses the SubDark dataset to train the PhotonFuse-Net plug-and-play end-to-end object detection framework. The GPU used for training is an NVIDIA A100 with 40GB of VRAM. The initial learning rate (lr0) is set to 0.01, the weight decay is set to 0.0005, the optimizer is set to AdamW, and the training epochs are set to 200. The SubDark dataset used in this paper consists of test sets collected in real, harsh low-light underground environments, which can rigorously test the model's generalization ability, robustness, and its ability to cope with challenges such as low light, shadows, and headlight glare, providing a reliable and comparable final benchmark for algorithm performance. Regarding the details of the object detection framework, the weights of both the illumination enhancement branch and the original image branch are set to 1, and the plug-and-play adaptive illumination enhancement module is replaced with the most advanced illumination enhancement network structures in existing research (ZeroDCE++, FeatEnhancer, IAT, CIDNet). To demonstrate the performance of the PhotonFuse-Net object detection framework after integrating an illumination enhancement network structure, this paper compares the experimental results of training object detection using the aforementioned illumination enhancement network structure enhanced with YOLOv8 (using default configuration), training object detection using an end-to-end model of the illumination enhancement network structure and YOLOv8, and training object detection using the plug-and-play adaptive illumination enhancement module of the PhotonFuse-Net framework with the illumination enhancement network structure from existing research. Furthermore, to demonstrate the superior performance of the PhotonFuse-Net object detection framework, this embodiment introduces experimental results from YOLOv8 (baseline), SSD, rt-detr, and MAET object detection frameworks (all using default settings) for comparison. The experimental results are shown in Table 3 below. Generally, a higher mAP is considered to indicate higher model recognition accuracy. Experimental results show that PhotonFuse-Net, with its plug-and-play modules replaced by ZeroDCE++, FeatEnhancer, IAT, and CIDNet, achieves mAP50 improvements of 4.2%, 4.8%, 4.1%, and 3.4% respectively compared to the baseline; and mAP50-95 improvements of 3%, 3.1%, 3.9%, and 4.1% respectively compared to the baseline. ZeroDCE++, IA, and CIDNet, compared to directly enhancing images with YOLOv8 and then performing object detection, improve mAP50 by 2.5%, 5.3%, and 1.8% respectively; and mAP50-95 improvements of 1.9%, 2.7%, and 2.4% respectively. Furthermore, the PhotonFuse-Net object detection framework outperforms the SSD, rt-detr, and MAET object detection frameworks on the SubDark dataset.The experimental results (as shown in Table 2) fully demonstrate the superiority of the PhotonFuse-Net object detection framework in low-light object detection in underground parking lots.

[0066] Table 2: Comparison of Experimental Results for the SubDark Dataset

[0067]

[0068] Analysis: This example analyzes the weights of the illumination enhancement branch and the original image branch in the framework. The parallel end-to-end framework (PhotonFuse-Net with noise suppression attention module removed) is still trained using the SubDark dataset. The plug-and-play adaptive illumination enhancement module uses ZeroDCE++. The weights of the illumination enhancement branch are... Weights of branches in the original graph The values ​​were set to 0.5, 1, 1.5, and 2 respectively, with the remaining configurations the same as in Section 5.1. The experimental results are shown in Table 3 below. The experimental results indicate that...

[0069] At that time, the mAP50 of the parallel end-to-end frame is the highest, which means that the PhotonFuse-Net frame achieves the best effect.

[0070] Table 3: Comparison of experimental results under different parameters in the SubDark dataset

[0071]

[0072] Ablation Study: This section will conduct an ablation study on the plug-and-play adaptive illumination enhancement module and noise suppression attention module in the PhotonFuse-Net framework based on the baseline model. The SubDark dataset was used for training, with the same training settings as in Section 5.1. The experimental results are shown in Table 4 below. Note that the parallel end-to-end framework in Table 4 means that the PhotonFuse-Net plug-and-play end-to-end object detection framework does not include the noise suppression attention module. The results demonstrate that the PhotonFuse-Net framework achieves optimal performance when both the plug-and-play adaptive illumination enhancement module and the noise suppression attention module are complete.

[0073] Table 4: Comparison of experimental results in ablation studies

[0074]

[0075] A target detection system based on multi-branch weighted fusion includes:

[0076] A dataset construction unit, used to perform step S1;

[0077] Model building unit, used to execute step S2;

[0078] The model training unit is used to perform step S3;

[0079] The detection unit is used to perform step S4.

[0080] The content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0081] A target detection device based on multi-branch weighted fusion:

[0082] At least one processor;

[0083] At least one memory for storing at least one program;

[0084] When the at least one program is executed by the at least one processor, the at least one processor implements a target detection method based on multi-branch weighted fusion as described above.

[0085] The content of the above method embodiments is applicable to the device embodiments. The specific functions implemented by the device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0086] A storage medium storing processor-executable instructions, which, when executed by a processor, are used to implement a target detection method based on multi-branch weighted fusion as described above.

[0087] The content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0088] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A target detection method based on multi-branch weighted fusion, characterized in that, Includes the following steps: Images under different lighting conditions are acquired and labeled to construct an image dataset. Based on the first branch, the second branch, and the fusion branch, a target detection model is constructed. The first branch includes an adaptive illumination enhancement module and a first feature extractor, and the second branch includes a second feature extractor; The target detection model is trained based on the image dataset to obtain the trained target detection model; The image to be tested is input into the trained target detection model to generate detection results; The step of inputting the image to be tested into the trained target detection model to generate detection results specifically includes: The image to be tested is input into the trained target detection model; Based on the first branch, illumination enhancement and feature extraction are performed on the image to be tested to obtain the first feature; Based on the second branch, feature extraction is performed on the image to be tested to obtain the second feature; Based on the fusion branch, feature fusion is performed on the first feature and the first feature, and a detection result is generated; The step of fusing the first feature and the first feature based on the fusion branch and generating a detection result specifically includes: The first feature and the second feature are weighted and added together to obtain the sum feature; The summation features are subjected to cross-dimensional interaction to obtain the interacted features; The sum of features is then focused on key regions to obtain the focused features; The features after interaction and the features after focusing are fused to obtain the final features; Detection results are generated based on the final features.

2. The target detection method based on multi-branch weighted fusion according to claim 1, characterized in that, The process of acquiring images under different lighting conditions and annotating them to construct an image dataset specifically includes: Images under different lighting conditions are acquired, and image enhancement is performed on the low-light images to obtain preprocessed images; The preprocessed images are labeled to obtain an image dataset.

3. The target detection method based on multi-branch weighted fusion according to claim 1, characterized in that, The cross-dimensional interaction of the summation and features is expressed by the following formula: in, Indicates the features after interaction. For the Sigmoid function, For a k×k convolutional layer in three branches, , , They are , The result obtained by performing Z-pooling on x. , These are the results obtained from rotating tensor x during CH and CW interactions, respectively.

4. The target detection method based on multi-branch weighted fusion according to claim 1, characterized in that, The key region focusing of the summed features is expressed by the following formula: in, Indicates the features after focusing. Represents the projection matrix; The superscript m indicates the m-th attention head; R represents position embedding; R represents relative position offset. This represents the output feature representation of the m-th attention head. This represents the query embedding of the m-th attention head. This represents the transpose of the transformed key embedding of the m-th attention head. This represents the feature dimension of each attention head. This represents the embedded value of the transformed value of the m-th attention head. Indicates input features, Represents the features after sampling. This represents a learnable linear projection matrix that projects input features onto the query space. This represents a learnable linear projection matrix that projects the sampled features onto the key space. This represents a learnable linear projection matrix that projects the sampled features onto the value space.

5. The target detection method based on multi-branch weighted fusion according to claim 1, characterized in that, During training, the loss function of the object detection model is as follows: in, This represents the bounding box regression loss. Represents classification loss, Indicates the distribution focus loss. This represents hyperparameters.

6. A target detection system based on multi-branch weighted fusion, characterized in that, For performing the target detection method based on multi-branch weighted fusion as described in claim 1, including: The dataset construction unit is used to acquire images under different lighting conditions and perform image annotation to construct an image dataset. The model building unit constructs an object detection model based on a first branch, a second branch, and a fusion branch; the first branch includes an adaptive illumination enhancement module and a first feature extractor, and the second branch includes a second feature extractor. The model training unit trains the target detection model based on the image dataset to obtain the trained target detection model. The detection unit inputs the image to be tested into the trained target detection model and generates detection results.

7. A target detection device based on multi-branch weighted fusion, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a target detection method based on multi-branch weighted fusion as described in any one of claims 1-5.

Images