Target detection method and system under low-light condition

Abstract

The invention discloses a target detection method and system under a low illumination condition, which are applied to the technical field of image processing, and are characterized in that a RepELAN is used as a main branch of a backbone network to extract features, and in each stage of the main branch, non-parameter downsampling is used to save details of image information; the main branch and the non-parameter down-sampling branch are subjected to learnable feature fusion, and the model can adaptively adjust the weight ratio of feature fusion; in the feature fusion network part, a GCGELAN structure is used for feature fusion, and high precision is ensured under the condition of low parameter quantity; on the basis of the PAFPN, Identity connection is used for a P4 feature layer and a P5 feature layer, and information loss in the forward process is made up; the LAHSFF is used to carry out adaptive feature fusion on the low-level feature layer to the high-level feature layer, and detail information in the high-level feature map is increased. And the detection precision of the real-time detector under the low-light condition is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more particularly to a method and system for detecting a target under low light conditions. Background Art

[0002] In recent years, the field of object detection has made significant progress and has been widely used in many fields with outstanding achievements. However, most detectors can achieve good detection results under normal lighting conditions with sufficient feature information. In low-light or dark environments, when feature information is insufficient, their detection accuracy is poor. Low-light scenes are inevitable in actual situations, such as at night, on rainy days, and in shadow conditions. Therefore, it is particularly important to strengthen the research on object detection in low-light scenes.

[0003] At present, the mainstream real-time detection algorithms mainly use single-stage detectors, such as the YOLO series based on convolutional neural networks (CNN). At the same time, detectors that integrate Transformer and Mamba are gradually being applied to real-time detection. The above detection algorithms usually extract features through a lightweight backbone network first, and then send the obtained feature maps of different scales to the feature fusion network for multi-scale feature fusion. A large number of studies have shown that the coordinated use of the backbone network and the feature fusion network can greatly improve the detection accuracy. The backbone network first extracts features from the original image, obtains feature maps of different scales at different stages, and then sends the obtained multi-scale feature maps to the feature fusion network for feature fusion of different scales. The use of the feature fusion network can greatly increase the detection accuracy, which is crucial to finally obtain accurate target semantic information.

[0004] When the backbone networks of the above detectors perform feature extraction, they use a downsampling method that increases the number of feature map channels to obtain features of different scales. After feature extraction is completed at each stage, feature extraction of the next stage is performed vertically upward. As the depth of the network increases, the network can capture higher-level semantic information. Generally, the semantic information extracted from low-level feature maps helps detect small targets, and the information extracted from high-level feature maps focuses on the detection of large targets. Although this feature extraction method can extract most of the effective information, it may lose some detailed information. Because in the process of downsampling, higher-level semantic information is obtained, and the most important information of the feature map of the previous stage relative to the current stage is retained, but some information useful for the next stage will be lost, which will reduce the final detection effect.

[0005] Among the existing feature extraction networks, the Feature Pyramid Network (FPN) is the most widely used. FPN fuses adjacent feature maps through top-down upsampling and lateral connections, and combines the characteristics of high-level feature maps with rich semantic information and low-level feature maps with rich location information to obtain new feature maps with richer semantic information. In order to make high-level feature maps have more accurate location information, a path aggregation network (PANet) was generated based on FPN. PANet performs well on a variety of data sets. PANet is also a feature fusion network widely used in most detectors.

[0006] However, this fusion method has the problem of inconsistency between scales. When there are both large and small targets in the image, after the feature maps of different sizes are fused, the focus on the targets will conflict. Targets with high-level features are regarded as important information, thereby ignoring the importance of targets with low-level features. This will affect the calculation of the gradient and reduce the detection accuracy of the final model.

[0007] Adaptive Spatial Feature Fusion (ASFF) filters the information of different feature maps and alleviates the semantic conflict caused by inconsistency between scales. Although ASFF can handle the problem of contradictory information at the same position, when aligning high-level feature maps to low-level feature maps, the semantic information of high-level feature maps is richer, the size of the target is larger, and it is easier to detect. Therefore, the low-level feature maps will regard the targets in the high-level feature maps as the main information. In addition, since the targets in the high-level feature maps are large, there is also a situation where the targets in the high-level feature maps are regarded as background by the low-level feature maps. To address these problems, based on the combination of PAFPN and ASFF, a forward spatial adaptive compensation feature fusion network (FSCFF) is proposed. After the high-level feature maps and the low-level feature maps are fully fused with each other, forward feature fusion is performed on the feature maps of each layer, and only the low-level feature images and high-level feature maps are adaptively spatially fused to prevent the high-level features from dominating the low-level feature maps.

[0008] Therefore, how to effectively improve the detection accuracy of real-time detectors under low-light conditions is an urgent problem that technicians in this field need to solve. Summary of the invention

[0009] In view of this, the present invention provides a method and system for target detection under low light conditions to solve the problems in the background technology.

[0010] In order to achieve the above object, the present invention adopts the following technical solution:

[0011] In one aspect, the present invention discloses a method for detecting an object under low light conditions, comprising:

[0012] Obtain low-light and dark-light data to build a low-light dataset;

[0013] The RFSC-Net detection model is trained using a low-light dataset to obtain a low-light detection model; wherein, the RFSC-Net detection model uses RepELAN as the main branch of the backbone network to extract features, and uses parameter-free downsampling to preserve the details of image information at each stage of the main branch; the main branch and the parameter-free downsampling branch perform learnable feature fusion, and adaptively adjust the weight ratio of feature fusion; in the feature fusion network part, the GCGELAN structure is used for feature fusion; on the basis of PAFPN, GCGELAN and the same-layer Identity connection are used; LAHSFF is used to adaptively fuse the low-level feature layer to the high-level feature layer;

[0014] The low-light image to be detected is input into the low-light detection model to obtain the low-light scene detection result.

[0015] Preferably, in the above-mentioned target detection method under low-light conditions, the main branch of the backbone network, RepELAN, adopts the multi-branch structure design of ELAN, and uses the multi-branch structure for feature extraction to capture features of receptive fields of different sizes. Based on the multi-branch design, RepEALN removes the 1×1 convolution in the first branch of ELAN, and performs multi-branch operation after convolution. DRepBottleneck uses two re-parameterized 3×3 convolutions for shortcut connection.

[0016] Preferably, in the above-mentioned target detection method under low-light conditions, SPD-Conv is set in each stage of the main branch; the SPD-Conv downsampling operation is performed before each downsampling convolution operation to retain more shallow detail information and compensate it to the high-level feature map.

[0017] Preferably, in the above-mentioned method for detecting an object under low light conditions, the specific steps of compensating to a high-level feature map are:

[0018] Get the feature map of the same input as the main branch;

[0019] Perform parameterless downsampling on the feature map to preserve shallow semantic information;

[0020] The final output feature map is fused with the output of RepELAN in the main branch.

[0021] Preferably, in the above-mentioned target detection method under low light conditions, the learnable feature fusion uses a fast normalization fusion method to perform feature fusion, and the weight normalization fusion method is as follows:

[0022]

[0023] where w j Yes i After the activation function SiLU, w i is a learnable parameter, I i is the i-th feature map to be fused, ε is a very small number set to prevent numerical instability, n is the number of feature maps to be fused, and p i It is the feature map output after fusion.

[0024] Preferably, in the above-mentioned target detection method under low-light conditions, on the basis of PAFPN, GCGELAN and the Identity connection of the same layer are used, and the two architectures of CSPNet and ELAN are combined. Internally, GELAN uses RepConv to replace ordinary convolution, uses RepNCSP structure, and uses heavy parameter convolution in CSP structure. The branches inside RepNCSP obtain multi-branch feature information and extract richer semantic information. The Global Context attention mechanism is used to perform feature screening and processing, and the feature map of Identity is first spatially attended and then serially connected to the channel attention.

[0025] Preferably, in the above-mentioned target detection method under low light conditions, the specific operation steps of the GCGELAN structure are:

[0026] After a 1×1 convolution, GC attention operation and GELAN operation are performed respectively;

[0027] After the operations of the two branches are completed, Concat splicing and 1×1 convolution are performed for feature fusion.

[0028] Preferably, in the above-mentioned method for detecting a target under low light conditions, the operation steps of using LAHSFF to adaptively fuse features from a low-level feature layer to a high-level feature layer are as follows:

[0029] LAHSFF first scales the feature maps of different layers to the same size;

[0030] Perform adaptive fusion operation on the scaled feature map;

[0031] The output size and number of channels are the same as the fused feature map of the input of this layer.

[0032] Preferably, in the above-mentioned target detection method under low light conditions, the feature map is adaptively fused, and the fusion formula is as follows:

[0033]

[0034] Among them, l represents the current corresponding layer number, k represents the feature map from the kth layer, and x ij k->l represents the feature map after the input from the kth layer is scaled to the size of the lth layer, where ij represents the coordinates of each spatial position in the feature map, α ij k->l Represents the weight of each position of the corresponding feature map, y ij l Represents the feature map output after feature fusion, which is obtained by fusion of the feature map of the lth layer; α ij k->l The calculation formula is as follows:

[0035]

[0036] The formula is completed through the Softmax activation function. After normalization, the value range of the weight coefficient is limited to the interval [0-1].

[0037] On the other hand, the present invention discloses a target detection system under low light conditions, applying the above method, comprising:

[0038] The data set acquisition module is used to acquire low-light and dark-light data to build a low-light data set;

[0039] The model training module is used to train the RFSC-Net model using the low-light dataset to obtain a low-light detection model;

[0040] The low-light detection module inputs the low-light image to be detected into the low-light detection model to obtain the low-light result.

[0041] Through the above technical solutions, it can be known that compared with the prior art, the present invention discloses a method and system for target detection under low-light conditions, which uses RepELAN as the main branch of the backbone network to extract features, reduce network overhead and improve network performance; at each stage of the main branch, non-parameter downsampling is used to preserve the details of image information; the main branch and the non-parameter downsampling branch perform learnable feature fusion, and the model can adaptively adjust the weight ratio of feature fusion; in the feature fusion network part, the GCGELAN structure is used for feature fusion to ensure high accuracy with low parameter quantity; on the basis of PAFPN, the P4 and P5 feature layers are connected with Identity to make up for the loss of information in the forward process; LAHSFF is used to adaptively fuse the low-level feature layer to the high-level feature layer to increase the detail information in the high-level feature map. This method effectively improves the detection accuracy of the real-time detector under low-light conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 It is a schematic diagram of the method steps of the present invention;

[0044] Figure 2 Schematic diagram of RFSC-Net structure;

[0045] Figure 3 (a) is a schematic diagram of the ELAN structure; (b) is a schematic diagram of the RepELAN structure;

[0046] Figure 4 (a) is a schematic diagram of the Bottleneck structure; (b) is a schematic diagram of the DRepBottleneck structure; (c) is a schematic diagram of the RepBottleneck structure;

[0047] Figure 5 (a) shows the structure of GELAN; (b) shows the use of GlobalContext (GC) block before Chunk; (c) shows the use of GC after Concat; (d) shows the schematic diagram of using GC in parallel with the main branch;

[0048] Figure 6 This is a RepConv example diagram;

[0049] Figure 7 It is a schematic diagram of the RepNCSP structure;

[0050] Figure 8 It is a schematic diagram of the GC structure;

[0051] Fig. 9 ASFFLH structure diagram;

[0052] Fig.10 Comparison chart of the results of RFSC-Net model detecting low-light dataset ExDark;

[0053] Fig.11 This is the heat distribution diagram of the attention of the RFSC-Net model on the feature map. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The network structure of RFSC-Net is as follows Figure 2 As shown in the figure, it consists of three parts: backbone network RSELAN, multi-scale feature fusion network FSCFF and Predict Head.

[0056] Embodiment 1:

[0057] The embodiment of the present invention discloses a low-light detection method, which improves the traditional YOLOv8n network structure. The steps of the detection method are as follows: Figure 1 Shown, including

[0058] Step 1: Obtain low-light scene data to build a low-light data set;

[0059] Step 2, using the low-light data set to train the RFSC-Net detection model to obtain a low-light detection model; wherein the RFSC-Net detection model uses RepELAN as the main branch of the backbone network to extract features, and uses parameter-free downsampling to preserve the details of image information at each stage of the main branch; the main branch and the parameter-free downsampling branch perform learnable feature fusion, and adaptively adjust the weight ratio of feature fusion; in the feature fusion network part, the GCGELAN structure is used for feature fusion; on the basis of PAFPN, GCGELAN and the same-layer Identity connection are used; LAHSFF is used to adaptively fuse the low-level feature layer to the high-level feature layer;

[0060] Step 3: Input the low-light image to be detected into the low-light detection model to obtain the low-light detection result.

[0061] The low-light dataset in step 1 includes several low-light images and the annotations corresponding to each low-light image.

[0062] In step 2, the low-light detection model uses the low-light dataset, takes the low-light image as input, and takes the annotation corresponding to the low-light image as output to train the RFSC-Net model.

[0063] The RFSC-Net model is based on the traditional YOLOv8 model, with innovations and improvements in the backbone network and feature fusion network. Its structure is as follows: Figure 2 As shown:

[0064] First, we improve the backbone network by adding the idea of ​​semantic compensation to the backbone network.

[0065] like Figure 3 As shown in the figure, RepELAN adopts a multi-branch structure design similar to ELAN. The design of the multi-branch structure allows the gradient to propagate from multiple paths, improves the fluidity of the gradient, reduces the risk of gradient disappearance, enhances the network's ability to process and receive gradients, and allows the network to become deeper. In addition, different branches learn different features during feature learning, which prevents the model from falling into a local optimal solution. Using a multi-branch structure for feature extraction can capture features of receptive fields of different sizes, increasing the diversity of feature learning. Based on the multi-branch design, RepEALN removes the 1×1 convolution in the first branch. Here it mainly refers to the C2f design of YOLOv8 and performs multi-branch operations after convolution. Compared with ELAN's use of ordinary 3×3 convolutions, re-parameterized convolutions are used, such as Figure 4 The Bottleneck shown in (a) uses two ordinary 3×3 convolutions, while Figure 4 (b) DRepBottleneck uses two re-parameterized 3×3 convolutions, followed by a shortcut connection. It is worth noting that the RepConv used does not have an Identity connection. Re-parameterized convolution can extract richer features during training, and multiple branches are fused into a single branch structure during inference, without increasing the inference time and computational cost.

[0066] Due to the lack of feature information of the target under low light conditions, directly using pooling operations for downsampling will result in reduced image information and fail to retain detail information well. Use SPD-Conv (Space-to-depth Convolution) for downsampling, perform as few operations on the feature map as possible, and retain the detail information of the low-level feature map to the greatest extent. After downsampling, 1×1 convolution is used to align the dimensions. By using SPD-Conv for downsampling operations, as much detail information as possible is retained at the cost of adding very few parameters. Figure 2 The specific structure and connection method of SPD-Conv are shown in RSELAN. SPD-Conv downsampling operation is performed before each downsampling convolution operation to retain more shallow detail information and compensate it to the high-level feature map, thereby adding more detail information to the high-level semantic feature map. For example, edge detail information and texture information of target features, which are not much detail information in low-light environments, are lost to a large extent after feature extraction. As the network deepens, detail information is seriously missing, which will reduce the final detection effect. The problem of detail loss is solved by gradually supplementing low-level semantic information to high-level feature maps.

[0067] Learning weight feature fusion method (LWFusion), such as Figure 2 As shown. When the compensation branch and the feature extraction module are combined, the importance of different feature maps is different. The common feature fusion method uses direct addition. The direct addition method does not take into account the importance of different feature maps. It is completely unnecessary to give high weights to feature maps with less contribution, which will result in less effective information contained in the fused feature map and reduce the performance of target detection. Therefore, the fast normalization fusion method is used for feature fusion. The weight normalization fusion method is as follows:

[0068]

[0069] where w j Yes i After the activation function SiLU, w i is a learnable parameter, I i is the i-th feature map to be fused, ε is a very small number set to prevent numerical instability, which is set to 0.0001 in this paper, n is the number of feature maps to be fused, and p i This is the feature map output after fusion. This normalization method limits the weight value to between [0-1], avoiding the appearance of outliers and making the training process more stable.

[0070] Then use GCGELAN and the Identity connection of the same layer in the PAFPN part;

[0071] Its structure is as Figure 5 As shown in the figure. The GELAN structure is a generalized efficient aggregation network that takes into account lightweight, inference speed and detection accuracy. The design idea of ​​its structure combines the two architectures of CSPNet and ELAN. Internally, GELAN uses RepConv to replace ordinary convolution to increase the richness of feature extraction. The use of the GELAN structure maintains a small number of parameters while ensuring accuracy. Inside GELAN, the RepNCSP structure is used. Figure 7 The detailed structure of RepNCSP is shown in the figure. RepNCSP uses a heavily parameterized convolution in the CSP structure, where the RepBottleneck structure is Figure 4 (c) Replace the first ordinary convolution in Bottleneck with RepConv, as shown in Figure 7 As shown in the figure, this only increases the training cost slightly compared to Bottleneck, and does not increase the inference cost.

[0072] The structural design of GELAN enables the branches inside RepNCSP to obtain multi-branch feature information and extract richer semantic information. However, the two branches that have not passed through the RepNCSP structure are connected by Identity and no feature transformation operation is performed. For these two branches, the retained information is more inclined to the original input information;

[0073] It should be noted that: the original feature map contains a lot of information. In the feature fusion stage, it is not necessary to extract as much information as possible like in the feature extraction network. Instead, we should pay attention to which features to fuse and how to fuse them. The issue of how to fuse has been considered in the GELAN network, but the features to be fused have not been screened. Therefore, a screening is performed on the input feature map to let the model focus on important information and ignore minor information. With this idea in mind, the Global Context (GC) attention mechanism is used to screen features. The feature map of Identity is first spatially attended to, and then the channel attention is serially connected. Three usage methods are designed. Figure 5 The specific location where GC is added is shown in the figure. Figure 8 This is the structural diagram of GC.

[0074] The top-down information flow path of the FPN network allows the low-level feature maps to have the high-level semantic information in the high-level feature maps, enriching the semantic information in the low-level feature maps. Based on the FPN network, PANet adds a bottom-up path to fuse the precise position information contained in the low-level feature maps into the high-level feature maps to make up for the lack of position information in the high-level feature maps. After the feature fusion operation of PAFPN, both the low-level feature maps and the high-level feature maps contain rich semantic information and position information. In the process of bidirectional feature fusion of PAFPN, due to the increase in propagation paths, the loss of features will also increase. Although it will be fused with the feature maps of the same scale in the FPN path each time, since the PANet path is deeper than the feature map initially extracted, only one fusion may not be able to make up for the feature loss. Therefore, in the three feature maps output by the bottom-up path of PAFPN, the two high-level feature maps output by PAFPN use Identity connection to increase the degree of fusion of the feature maps output by PAFPN. See the connection method. Figure 1 The feature fusion operation used is the same as that of the feature extraction network, and the LWFusion method is used to achieve a greater degree of feature fusion at a minimal cost.

[0075] Finally, ASFFLH is used on the three outputs of PAFPN to supplement the semantic information of the low-level feature maps to the high-level feature maps;

[0076] The output of PAFPN is used as the input of ASFFLH. The flowchart of LAHSFF is shown in Figure 9. ASFFLH first scales the feature maps of different layers to the same size, and then performs adaptive fusion operation on the scaled feature maps. The size and number of channels of the feature maps output after adaptive fusion are the same as the input of this layer. First, the feature map alignment operation is performed. Since the data flow of feature fusion is only from bottom to top, that is, the large-scale feature map compensates the information of the small-scale feature map, there is only downsampling when the feature map is scaled, and there is no upsampling. For the feature map that is downsampled twice, this paper uses a 3×3 convolution kernel with a step size of 2. For the feature map that is downsampled four times, the maximum pooling is first used for scaling, and then the convolution kernel with a step size of 2 and a size of 3×3 is used for downsampling. After scaling the feature map size, the channels are aligned and all feature maps are compressed. A common practice is used here. The convolution kernel with a size of 1×1 and a step size of 1 is directly used to compress the number of channels of all feature maps to the same number. The number is set in the same way as the implementation in ASFF, using 8 channels. The feature map after feature scaling is adaptively fused, and the fusion formula is as follows:

[0077]

[0078] Among them, l represents the current corresponding layer number, k represents the feature map from the kth layer, and x ij k->l represents the feature map after the input from the kth layer is scaled to the size of the lth layer, where ij represents the coordinates of each spatial position in the feature map, α ij k->l Represents the weight of each position of the corresponding feature map, y ij l Represents the feature map output after feature fusion, which is obtained by fusion of the feature map of the lth layer. ij k->l The calculation formula is as follows:

[0079]

[0080] The formula is completed through the Softmax activation function. After normalization, the value range of the weight coefficient is limited to the interval [0-1], and the training process becomes more stable.

[0081] When l=0, the shallowest feature map is obtained, and no feature fusion is performed with other layers, so the above adaptive fusion operation is not performed for feature compensation. Combined with the characteristics of the 0th layer feature map, the feature fusion degree of the low-level feature map is low. In the PAFPN process, only one GCGELAN fusion process is performed, and the feature map of this layer may have insufficient feature fusion. Feature fusion is performed again in the 0th layer of LAHSFF to enhance the fusion degree of feature information. Considering that the target in the low-level feature map is small, retaining more detail information helps to improve detection accuracy, so no attention operation is used during fusion.

[0082] The success of the Transformer model proposed in MetaFormer is attributed to the specific architecture. In addition, in the process of model design, the design of existing feature extraction networks and feature fusion networks generally follows the unity of the overall structure and uses repeated blocks for stacking. Therefore, in order to maintain the unity of the structure with PAFPN, the module used in the 0th layer is the same as the post-Concat module in PAFPN. For example, if this paper uses GCGELAN in PAFPN, then the fusion module used in the 0th layer of LAHSFF is GELAN without GC attention.

[0083] The feature map after the LAHSFF module once again integrates the shallow feature map, thereby enriching the feature information in the high-level feature map. For the lowest-level feature map, the features in the feature map are more fully integrated after GELAN.

[0084] Embodiment 2:

[0085] This embodiment performs target detection on low-light images through specific experiments to verify the beneficial effects of the method of the present invention.

[0086] Experimental platform: The operating system of the experimental platform is Windows 11 (Professional Edition), the CPU is Inter(R) Core(TM) i5-10400F@2.9GHz, the memory is 32GB, the graphics card is Nvidia GeForce RTX 2080Ti, and the video memory is 22GB. The CUDA version used by torchvision is 11.8, the deep learning framework is Pytorch-GPU 2.1.0+cu118, and the deep neural network GPU acceleration library is cuDNN8.5.0. The training process is as follows Figure 5 shown.

[0087] Implementation details: The evaluation experiment of this embodiment is conducted on ExDark, and generalization experiments are conducted on VisDrone, DarkFace, COCO 2017 and PASCAL VOC2007+2012. Table 1 shows the training set, validation set and number of categories of each dataset. All models are retrained, and the training is unified for 300 epochs, and the batch size is set to 32. The optimizer uses the stochastic gradient descent algorithm (SGD) with weight decay, the weight decay rate is set to 0.005, the initial learning rate is set to 0.01, and warm-up is used to warm up the learning rate in the first three epochs, and the subsequent learning rate adopts linear changes. In the pre-processing, Mosiac, MixUp, Random HSV, RandsomAffineh and HorizontalFlip data enhancement methods are performed, and Mosiac data enhancement is turned off in the last 10 epochs.

[0088] Table 1: Division and categories of training set and validation set

[0089]

[0090] Comparative experimental results:

[0091] Comparison of lightweight feature extraction networks: Table 2 shows the comparison of RSELAN with different lightweight backbone networks in RFSC-Net. Compared with other backbone networks trained from scratch, RSELAN achieved the highest accuracy of 44.0% and 69.3% in RFSC-Net. In the comparative experiment, GhostNetV21.0× is the backbone network with the highest detection accuracy based on the CNN method. RSELAN is 0.4% and 0.6% higher than GhostNetV21.0× in mAP and AP50, respectively, while the overall model parameters are less than half of GhostNetV21.0×, and the computation is 0.2G lower. Compared with MobileNetV20.5×, which has the lowest parameters and computation, RSELAN improves mAP and AP50 by 1.3% and 1.7% respectively with only 0.4M parameters. Among the lightweight Vision Transforms, RepViT and MobileViT are the two backbone networks with better effects. When the number of parameters is similar, RSELAN is 0.3% and 0.4% higher than MobileViT-XXS in mAP and AP50, respectively, while the amount of computation is reduced by 1.6G. RSELAN achieves the highest accuracy when the number of parameters and computation is comparable to other backbone networks.

[0092] Table 2: Comparative experiments of different backbones

[0093]

[0094] Comparison of multi-scale feature fusion: Table 3 shows the comparison of FSCFF and the currently commonly used multi-scale feature fusion methods in RFSC-Net. Compared with other feature fusion methods, the highest mAP and AP50 of 44.0% and 69.3% are achieved on the low-light dataset ExDark. Compared with PA-FPN, the mAP and AP50 of FSCFF are increased by 2.0% and 1.5%, respectively, and the number of parameters is only increased by 0.3M. ASFF is a combination of FPN+ASFF. Compared with ASFF, the mAP of FSCFF is 3.9% higher and the AP50 is 3.3\% higher. BiFPN only processes feature maps of three scales and removes the P6 and P7 layers. Compared with BiFPN, the mAP and AP50 of FSCFF are both increased by 1.2%. Compared with ResGFPN, the mAP and AP50 of FSCFF are increased by 1.8% and 0.9%, respectively. The accuracy of FSCFF is greatly improved with a small increase in the number of parameters and calculations.

[0095] Table 3: Comparative experiments of different feature fusion networks

[0096]

[0097] Comparison of attention mechanisms: Table 4 shows the effects of using different attention mechanisms in GCGELAN. Four types of attention mechanisms were used for comparison, namely channel attention, coordinate attention, serial channel spatial attention, and parallel channel spatial attention. The results show that the attention mechanism plays a very important role in the final detection accuracy. Using a suitable attention method will greatly improve the recognition accuracy. Among the four types of attention, channel attention, coordinate attention, and serial channel spatial attention have poor effects, and the accuracy has dropped significantly compared to not using the attention mechanism. The parallel BAM effect is better than the other three, but the mAP and AP50 still show a decrease of 0.5% and 0.7%. GC is the only one among all attention that exceeds RFSC-Net without attention. It is speculated that this is because GC can capture more global information, and the captured global information helps to improve the detection accuracy under low light conditions.

[0098] Table 4: Comparison of different attention in GCGELAN

[0099]

[0100]

[0101] Comparison with the latest technology: Table 5 shows the comparison of RFSC-Net with other real-time detectors trained from scratch. While maintaining the same number of parameters as YOLOv8n, RFSC-Net's mAP and AP50 increased by 2% and 2.1% respectively compared to the baseline, and the inference speed increased by only 0.4ms. Compared with YOLOv5s, RFSC-Net's mAP increased by 6.0%, and the latency was only 43.0% of YOLOv5n. Among the compared models, YOLOv8s has the highest accuracy, and RFSC-Net's mAP is still 0.3% higher than YOLOv8s. In summary, compared with the baseline, RFSC-Net only increased the amount of calculation by 2.0G and the inference time by 0.4ms without increasing the number of parameters, achieving the highest accuracy among all models.

[0102] Table 5: Comparative experiments with state-of-the-art real-time detectors

[0103]

[0104]

[0105] In this embodiment, a large number of ablation experiments are conducted on RFSC-Net to prove the role of its components in low-light detection. The ablation experiments are as follows:

[0106] Ablation between components of RFSC-Net: The structure of YOLOv8n is used as the baseline model to verify the effect of different structures relative to the baseline model, as shown in Table 6, where each structure is replaced on the basis of YOLOv8n. The experimental results show that although the mAP of RSELAN has not improved, the number of parameters of the model has been reduced, and the AP50 has increased by 0.8%. The use of GCGELAN and RSELAN has improved both mAP and AP50. From the results, it can be seen that the effect of using FSCFF* with GCGELAN is greater than that of using them independently. In general, different combinations contribute to the improvement of accuracy, but the combination of the three has the best effect, reaching 44.0% mAP and 69.3% AP50.

[0107] Table 6: Ablation between components of RFSC-Net

[0108]

[0109]

[0110] Ablation of GC at different positions in GCGELAN: Ablation experiments were conducted on GC at different positions in GCGELAN. As shown in Table 7, the effect before ResNCSP of GCGLEAN is the worst. Compared with not using GC, AP50 is reduced by 0.3%. After GC is added to ResNCSP, mAP and AP50 increase by 1.3% and 1.6% respectively. However, due to the processing of all branches, the number of parameters and the amount of calculation increase. It is believed that the attention operation on the features that pass through the RepNCSP branch will lead to over-learning and information loss. In order to avoid this information loss, a method is designed to perform attention operation only on the branches that do not pass through RepNCSP. From the experimental results, the speculation of this embodiment is verified. Both mAP and AP50 have been greatly improved. At the same time, compared with the case of After, the number of parameters is reduced by 0.2M and the amount of calculation is reduced by 0.2G.

[0111] Table 7: Ablation of GC at different locations in GCGELAN

[0112]

[0113] Ablation of ASFFLH module: As shown in Table 8, the results of using ASFFLH and ASFF in RFSC-Net show that the accuracy of using ASFF is 0.1% lower than that of direct prediction, which verifies the previous speculation that the fusion of high-level semantic information to the underlying semantics will cause semantic conflicts, thereby affecting the prediction of this layer. Due to the use of adaptive feature fusion, the impact of semantic conflicts is small, but the number of parameters and calculations increases significantly. The use of ASFFLH for fusion into mAP is 1.1% higher than that without using it, and AP50 is increased by 0.8%. The number of parameters and calculations are the same as using ASFF, which verifies that the semantic supplementation of the underlying semantics to the high-level feature map contributes greatly to the improvement of the final detection accuracy.

[0114] Table 8: Ablation of ASFFLH module

[0115]

[0116] Ablation of different blocks in the 0th layer of ASFFLH: As shown in Table 9, in the 0th layer of ASFFLH, the commonly used blocks in some current lightweight networks are used for experiments. Obviously, from the experimental results, GELAN has the highest accuracy. Using blocks different from those in the feature fusion network is not conducive to improving the detection accuracy. It verifies that using blocks with the same overall structure can maximize the effect of ASFFLH.

[0117] Table 9: Ablation of different blocks in ASFFLH layer 0

[0118]

[0119] Generalization test: In order to verify the generalization of RFSC-Net, this paper conducts experiments on COCO 2017, PASCAL VOC 2007+2012, VisDrone and DarkFace. The experimental results are shown in Table 10. For the detection of the small target dataset VisDrone, RFSC-Net improves the mAP by 1.6% over the baseline. On the low-light small target dataset DarkFace, the mAP of RFSC-Net is also improved by 0.4% compared with the baseline. In addition, on COCO 2017 and PASCAL VOC 2007+2012, RFSC-Net is 1.9% and 2.5% higher than the baseline mAP, respectively. It can be seen that RFSC-Net is not only effective for low-light detection, but also has significant improvements on the more difficult DarkFace dataset.

[0120] Table 10: Performance of RFSC-Net on different datasets

[0121]

[0122] Another embodiment of the present invention further discloses a low-light pedestrian detection system, comprising:

[0123] A data set acquisition module is used to acquire low-light data and build a low-light data set;

[0124] The model training module is used to train the improved YOLOv8 model RFSC-Net using a dense pedestrian dataset to obtain a low-light detection model;

[0125] The low-light detection module inputs the low-light image to be detected into the low-light detection model to obtain the detection result.

[0126] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0127] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for object detection under low light conditions, It is characterized in that include: Obtain low-light and dark-light data to build a low-light dataset; The RFSC-Net detection model is trained using a low-light dataset to obtain a low-light detection model; wherein, the RFSC-Net detection model uses RepELAN as the main branch of the backbone network to extract features, and uses parameter-free downsampling to preserve the details of image information at each stage of the main branch; the main branch and the parameter-free downsampling branch perform learnable feature fusion, and adaptively adjust the weight ratio of feature fusion; in the feature fusion network part, the GCGELAN structure is used for feature fusion; on the basis of PAFPN, GCGELAN and the same-layer Identity connection are used; LAHSFF is used to adaptively fuse the low-level feature layer to the high-level feature layer; The low-light image to be detected is input into the low-light detection model to obtain the low-light scene detection result.

2. The method for detecting a target under low light conditions according to claim 1, It is characterized in that The main branch of the backbone network, RepELAN, adopts the multi-branch structure design of ELAN, and uses the multi-branch structure for feature extraction to capture the features of receptive fields of different sizes. Based on the multi-branch design, RepEALN removes the 1×1 convolution in the first branch of ELAN, and performs multi-branch operations after convolution. DRepBottleneck uses two re-parameterized 3×3 convolutions for shortcut connection.

3. The method for detecting a target under low light conditions according to claim 1, It is characterized in that SPD-Conv is set up at each stage of the main branch; SPD-Conv downsampling operation is performed before each downsampling convolution operation to retain more shallow detail information and compensate it to the high-level feature map.

4. The method for detecting a target under low light conditions according to claim 3, It is characterized in that Specific steps to compensate to high-level feature maps: Get the feature map of the same input as the main branch; Perform parameterless downsampling on the feature map to preserve shallow semantic information; The final output feature map is fused with the output of RepELAN in the main branch.

5. The method for detecting a target under low light conditions according to claim 1, It is characterized in that Learnable feature fusion uses fast normalization fusion to perform feature fusion. The weight normalization fusion method is as follows: where w j Yes i After the activation function SiLU, w i is a learnable parameter, I i is the i-th feature map to be fused, ε is a very small number set to prevent numerical instability, n is the number of feature maps to be fused, and p i It is the feature map output after fusion.

6. The method for detecting a target under low light conditions according to claim 1, It is characterized in that On the basis of PAFPN, GCGELAN and Identity connection at the same layer are used, combining the two architectures of CSPNet and ELAN. Internally, GELAN uses RepConv to replace ordinary convolution, uses the RepNCSP structure, and uses heavy parameter convolution in the CSP structure. The branches inside RepNCSP obtain multi-branch feature information and extract richer semantic information. The GlobalContext attention mechanism is used for feature screening and processing. The feature map of Identity is first spatially attended, and then the channel attention is serially connected.

7. The method for detecting a target under low light conditions according to claim 1, It is characterized in that The specific steps of the GCGELAN structure are: After a 1×1 convolution, GC attention operation and GELAN operation are performed respectively; After the operations of the two branches are completed, Concat splicing and 1×1 convolution are performed for feature fusion.

8. The method for detecting a target under low light conditions according to claim 1, It is characterized in that The steps of using LAHSFF to adaptively fuse the low-level feature layer to the high-level feature layer are as follows: LAHSFF first scales the feature maps of different layers to the same size; Perform adaptive fusion operation on the scaled feature map; The output size and number of channels are the same as the fused feature map of the input of this layer.

9. The method for detecting a target under low light conditions according to claim 1, It is characterized in that The feature map performs adaptive feature fusion, and the fusion formula is as follows: Among them, l represents the current corresponding layer number, k represents the feature map from the kth layer, and x ij k->l represents the feature map after the input from the kth layer is scaled to the size of the lth layer, where ij represents the coordinates of each spatial position in the feature map, α ij k->l Represents the weight of each position of the corresponding feature map, y ij l Represents the feature map output after feature fusion, which is obtained by fusion of the feature map of the lth layer; α ij k->l The calculation formula is as follows: The formula is completed through the Softmax activation function. After normalization, the value range of the weight coefficient is limited to the interval [0-1].

10. A target detection system under low light conditions, using the target detection method under low light conditions according to any one of claims 1 to 9, It is characterized in that include: The data set acquisition module is used to acquire low-light and dark-light data to build a low-light data set; The model training module is used to train the RFSC-Net model using the low-light dataset to obtain a low-light detection model; The low-light detection module inputs the low-light image to be detected into the low-light detection model to obtain the low-light result.

Images