Low-illumination target detection method and device based on YOLOv7-tiny improvement

Abstract

The invention provides a low-illumination target detection method and device based on YOLOv7-tiny improvement, and relates to the technical field of image detection. A preset data set is obtained and preprocessed to determine a training set, a verification set and a test set; introducing a preset backbone network part and a preset neck structure into a preset YOLOv7-tiny detection model, and coordinating coordinate convolution to generate a low-illumination target detection model; training the low-illumination target detection model to generate a target detection model; and detecting the to-be-detected image information and obtaining a detection result. According to the method, the low-illumination target detection model is used for detection, the capturing capability of the low-illumination target detection model on target details under the low-illumination condition is improved, so that the problem caused by insufficient illumination is relieved, and the performance and accuracy of target detection under the low-illumination environment are effectively improved by improving the structure of the preset YOLOv7-tiny detection model.

Description

Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a low-illumination target detection method and device based on an improvement of YOLOv7-tiny. Background Art

[0002] The target detection task under low illumination refers to the task of detecting and identifying target objects in an environment with dim or insufficient lighting conditions.

[0003] In the prior art, a target detector is usually used to detect images under low illumination. However, due to insufficient lighting during shooting, the details of the target object in the image may not be clear. For the above problem, the detection accuracy of the existing target detector is low.

[0004] Therefore, how to effectively improve the detection accuracy of images under low illumination has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to effectively improve the detection accuracy of images under low illumination, the present application provides a low illumination target detection method, system, device and storage medium improved based on YOLOv7-tiny.

[0006] In the first aspect, the present application provides an improved low-light target detection method based on YOLOv7-tiny, which adopts the following technical solutions:

[0007] An improved low-light target detection method based on YOLOv7-tiny, comprising:

[0008] Obtain a preset data set, and preprocess the preset data set to determine a training set, a validation set, and a test set;

[0009] Introduce the preset backbone network part, preset neck structure and coordinated coordinate convolution into the preset YOLOv7-tiny detection model to generate a low-light target detection model;

[0010] Training the low-light target detection model based on the training set and the validation set to generate a target detection model;

[0011] Acquire the image information to be detected, detect the image information to be detected according to the target detection model and obtain the detection result.

[0012] Optionally, the step of obtaining a preset data set and preprocessing the preset data set to determine a training set, a validation set, and a test set includes:

[0013] Acquire a preset data set, and determine a target low-light image set and a target bounding box in the preset data set;

[0014] Convert the target low-light image set and the target bounding box into a YOLO training format to obtain a conversion result;

[0015] A preset division ratio is obtained, and the conversion result is divided according to the preset division ratio to determine a training set, a validation set, and a test set.

[0016] Optionally, before the step of introducing a preset backbone network part, a preset neck structure, and a coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model, the step further includes:

[0017] Introducing an efficient layer aggregation network module integrating multiple branch blocks, and generating a preset backbone network part according to the efficient layer aggregation network module integrating multiple branch blocks;

[0018] Introducing a cross-level network module of an attention mechanism, and constructing a preset neck structure according to the cross-level network module of the attention mechanism, wherein the cross-level network module of the attention mechanism includes a deformable convolution DCNv2 module and a CBAM attention mechanism module;

[0019] A coordinated coordinate convolution is introduced, wherein the structure of the coordinated coordinate convolution includes coordinate definition, coordinate transformation, coordinate and feature map concatenation and convolution operation.

[0020] Optionally, the step of introducing a preset backbone network part, a preset neck structure, and coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model includes:

[0021] The efficient layer aggregation network module integrating the multi-branch blocks is adopted in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks;

[0022] Obtaining a feature fusion module of the neck portion corresponding to the improved efficient feature extraction network with multi-branch blocks, and replacing the cross-level network module of the attention mechanism to obtain an improved Neck network;

[0023] The convolution layer before upsampling in the improved Neck network is replaced by the coordinated coordinate convolution to construct the low-illumination target detection model.

[0024] Optionally, the step of adopting the efficient layer aggregation network module integrating the multi-branch block in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks includes:

[0025] Get the network structure of the backbone network in the preset YOLOv7-tiny detection model;

[0026] The network structure is replaced by the efficient layer aggregation network module of the fused multi-branch block, so as to obtain output feature maps of different layers through the efficient layer aggregation network module of the fused multi-branch block during the forward propagation process;

[0027] Get the current replaced network structure and generate an efficient feature extraction network with multi-branch blocks.

[0028] Optionally, the step of obtaining a feature fusion module of the neck portion corresponding to the improved efficient feature extraction network with a multi-branch block, and replacing the cross-level network module of the attention mechanism to obtain an improved Neck network includes:

[0029] Obtaining a feature fusion module for the neck portion corresponding to the improved efficient feature extraction network with multi-branch blocks;

[0030] Replace the cross-level network module of the attention mechanism with the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks;

[0031] Performing a convolution operation on the acquired input information through the first convolution layer to obtain a first result;

[0032] Using the first result as input of a second convolutional layer, and performing a convolution operation on the first result through the second convolutional layer to obtain a second result;

[0033] Pass the first result through the GS bottleneck module and the depth-separable convolutional layer respectively to obtain a third result;

[0034] splicing the second result and the third result to obtain a splicing feature;

[0035] An improved Neck network is obtained based on the splicing features and the CBAM attention mechanism module.

[0036] Optionally, the step of replacing the convolution layer of the previous upsampling layer in the improved Neck network with the coordinated coordinate convolution to construct the low-illumination target detection model includes:

[0037] Before the upsampling operation in the improved Neck network, obtaining an input feature map;

[0038] Defining each pixel position of the input feature map as a coordinate vector;

[0039] Splicing the converted coordinate vector with the input feature map to obtain a spliced ​​feature map;

[0040] The convolution operation is performed on the spliced ​​feature map using the convolution kernel of the coordinated coordinate convolution to construct the low-illumination target detection model.

[0041] In a second aspect, the present application provides a low-light target detection system based on an improvement of YOLOv7-tiny, comprising:

[0042] A preset data set acquisition module is used to acquire a preset data set and preprocess the preset data set to determine a training set, a validation set, and a test set;

[0043] A low-light target detection model generation module is used to introduce a preset backbone network part, a preset neck structure, and coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model;

[0044] A target detection model generation module, used to train the low-light target detection model based on the training set and the verification set to generate a target detection model;

[0045] The detection result generation module is used to obtain the image information to be detected, detect the image information to be detected according to the target detection model and obtain the detection result.

[0046] In a third aspect, the present application provides a computer device, comprising: a memory and a processor, wherein the processor executes the method described above when running computer instructions stored in the memory.

[0047] In a fourth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0048] In summary, the present application includes the following beneficial technical effects:

[0049] This application obtains a preset data set and preprocesses the preset data set to determine the training set, validation set and test set; introduces a preset backbone network part, a preset neck structure and a coordinated coordinate convolution in the preset YOLOv7-tiny detection model to generate a low-light target detection model; trains the low-light target detection model based on the training set and the validation set to generate a target detection model; obtains the image information to be detected, detects the image information to be detected according to the target detection model and obtains the detection result. The low-light target detection model is used for detection to improve the low-light target detection model's ability to capture target details under low-light conditions, thereby alleviating the problem caused by insufficient lighting. In addition, by improving the preset YOLOv7-tiny detection model structure, the performance and accuracy of target detection in low-light environments are effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0051] Figure 2 It is a flowchart of the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application.

[0052] Figure 3 It is a structural diagram of an efficient layer aggregation network module of the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application, which integrates multiple branch blocks;

[0053] Figure 4 This is the DiverseBranchBlock structure diagram of the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application;

[0054] Figure 5 It is a structural diagram of the VoVGSCSP-DCN-CBAM module improved in the first embodiment of the low-illumination target detection method improved based on YOLOv7-tiny of the present application;

[0055] Figure 6 This is a structural diagram of the CBAM attention mechanism module introduced in the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application;

[0056] Figure 7 It is a structural diagram of the CoordConv module of the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application;

[0057] Figure 8 This is the overall network structure diagram of the improved model of the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application;

[0058] Fig. 9 It is a structural block diagram of the first embodiment of the low-light target detection system improved based on YOLOv7-tiny in this application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] Reference Figure 1 , Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0061] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0062] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0063] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a low-light target detection program improved based on YOLOv7-tiny.

[0064] exist Figure 1In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the present application can be set in the computer device, and the computer device calls the improved low-light target detection program based on YOLOv7-tiny stored in the memory 1005 through the processor 1001, and executes the improved low-light target detection method based on YOLOv7-tiny provided in the embodiment of the present application.

[0065] The present application embodiment provides a low-light target detection method based on YOLOv7-tiny improvement, referring to Figure 2 , Figure 2 This is a flow chart of the first embodiment of the low-light target detection method improved based on YOLOv7-tiny in this application.

[0066] In this embodiment, the low-light target detection method improved based on YOLOv7-tiny includes the following steps:

[0067] Step S10: Obtain a preset data set, and preprocess the preset data set to determine a training set, a validation set, and a test set.

[0068] It should be noted that the preset data set in this embodiment is a sample data set, including the public sample data ExDark data set, and its labels are converted into the YOLO training format. Specifically, the YOLO training format usually requires that the label of each sample contains the category identifier of the target and the location information of the bounding box. Specifically, the label format commonly used by YOLO is "category ID×center point coordinates×bounding box width×bounding box height". During the conversion process, the label information of the original sample data set needs to be parsed and reorganized and saved according to the label format of YOLO.

[0069] Furthermore, in order to improve the practicality in the low-light target detection process, a preset data set is obtained, and the preset data set is preprocessed to determine the training set, the verification set and the test set, including: obtaining the preset data set, determining the target low-light image set and the target bounding box in the preset data set; converting the target low-light image set and the target bounding box into the YOLO training format to obtain the conversion result; obtaining a preset division ratio, and dividing the conversion result according to the preset division ratio to determine the training set, the verification set and the test set.

[0070] In a specific implementation, the training set is used for model training, the validation set is used to evaluate the performance of the model during the model training process, and the test set is used to evaluate the performance of the model when the model processes untouched data. The training set, validation set, and test set can be divided in a ratio of 7:1:2. The sample dataset can contain 7,363 low-light images of 10 different conditions from very low-light environments to dusk; and target bounding boxes of 12 target classes at the image class level and local annotations.

[0071] Step S20: Introduce a preset backbone network part, a preset neck structure, and a coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model.

[0072] It should be noted that the multi-scale context aggregation backbone network is a backbone network structure that aims to improve the performance of target detection tasks through multi-scale feature information and context information. However, due to the presence of noise and blurred details in low-light scenes, the performance of the multi-scale context aggregation backbone network is poor. In order to improve the ability of the low-light target detection model to capture target details under low-light conditions, thereby alleviating the problems caused by insufficient lighting, in this embodiment, the YOLOv7-tiny target detection model is used and improved.

[0073] In a specific implementation, before the step of introducing a preset backbone network part, a preset neck structure and coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model, it also includes: introducing an efficient layer aggregation network module that integrates multiple branch blocks, and generating a preset backbone network part according to the efficient layer aggregation network module that integrates multiple branch blocks; introducing a cross-level network module of an attention mechanism, and constructing a preset neck structure according to the cross-level network module of the attention mechanism, wherein the cross-level network module of the attention mechanism includes a deformable convolution DCNv2 module and a CBAM attention mechanism module; introducing coordinated coordinate convolution, wherein the structure of the coordinated coordinate convolution includes coordinate definition, coordinate transformation, coordinate and feature map splicing and convolution operation.

[0074] Furthermore, in order to realize the generation of a low-illumination target detection model, a preset backbone network part, a preset neck structure and a coordinated coordinate convolution are introduced into a preset YOLOv7-tiny detection model to generate a low-illumination target detection model, including: using an efficient layer aggregation network module that fuses multi-branch blocks in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks; obtaining a feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks, and replacing the cross-level network module of the attention mechanism to obtain an improved Neck network; replacing the convolution layer of the previous layer of upsampling in the improved Neck network with a coordinated coordinate convolution to construct a low-illumination target detection model.

[0075] It should be noted that the steps of using an efficient layer aggregation network module that integrates multi-branch blocks in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks include: obtaining the network structure of the backbone network in the preset YOLOv7-tiny detection model; replacing the network structure with the efficient layer aggregation network module that integrates multi-branch blocks, so as to obtain the output feature maps of different layers through the efficient layer aggregation network module that integrates multi-branch blocks during the forward propagation process; obtaining the current replaced network structure and generating an efficient feature extraction network with multi-branch blocks.

[0076] It should be noted that the current replaced network structure is obtained and an efficient feature extraction network with a multi-branch block is generated, wherein the generated efficient feature extraction network is composed of 4 efficient layer aggregation networks fused with multi-branch blocks and some other structures.

[0077] It is understandable that the backbone network of the single-stage target detection model YOLOv7-tiny is improved by using the ELAN-DBB (Efficient Layer Aggregation Network with Multi-branch Blocks) module to replace the original MCB structure of the backbone network. Figure 3 As shown. ELAN-DBB introduces a multi-branch structure with different receptive fields and different complexities, thereby enriching the feature space and extracting more low-light image information. The DBB (DiverseBranchBlock) module adopts multiple branch structures. Each branch has a different receptive field and feature extraction capability, capturing feature information of different scales and levels, and can better adapt to the diversity of low-light target detection. The multi-branch structure can capture richer feature information, and feature fusion can improve the feature representation ability, enabling the model to detect more accurately. At the same time, it can also increase the complexity and expression ability of the network, thereby reducing the risk of overfitting and improving the generalization ability of the model. The DiverseBranchBlock structure diagram is shown below Figure 4As shown in the figure. In the forward propagation process of the ELAN-DBB model, a feature fusion process is formed by connecting the outputs of different layers. In this model, a feature fusion input is created by connecting the x1 layer processed by the DiverseBranchBlock module with other x2, x4 and x6 layers. This feature fusion can help the model comprehensively use features of different levels and receptive fields, thereby improving the accuracy and robustness of low-light target detection.

[0078] It can be understood that in deep learning, forward propagation refers to the transfer and calculation process of input data from the input layer to the output layer through the neural network model. In the object detection task, the forward propagation process refers to the process of processing the input image through the object detection model to obtain the location and category information of the object.

[0079] It should be noted that the steps of obtaining the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks, and replacing the cross-level network module of the attention mechanism to obtain the improved Neck network include: obtaining the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks; replacing the cross-level network module of the attention mechanism with the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks; performing a convolution operation on the obtained input information through the first convolution layer to obtain a first result; using the first result as the input of the second convolution layer, and performing a convolution operation on the first result through the second convolution layer to obtain a second result; passing the first result through the GS bottleneck module and the depth-separable convolution layer respectively to obtain a third result; splicing the second result and the third result to obtain a splicing feature; and obtaining the improved Neck network according to the splicing feature and the CBAM attention mechanism module.

[0080] In the specific implementation, a new neck structure is introduced in the feature fusion part of the model, that is, the original feature fusion module is replaced with the VoVGSCSP-DCN-CBAM module; the VoVGSCSP-DCN-CBAM module combines the deformable convolution DCNv2 module and the CBAM attention mechanism module on the basis of VoVGSCSP. The input of the module first passes through a convolution layer, then passes through the GSBottleneck module and the DCNv2 module, and the output of the DCNv2 module and the output after convolution again through the first convolution layer are spliced, and the channel dimension is reduced, and finally passes through the CBAM attention mechanism module.

[0081] It should be noted that for the single-stage target detection model YOLOv7-tiny's neck feature fusion network PANet structure, a new neck structure is designed based on the improved design paradigm SlimNeck, such as Figure 8 As shown. In order to speed up the prediction calculation, the feed image in CNN usually needs to go through a similar transformation process, in which the spatial information is gradually transferred to the channel. However, each spatial compression and channel expansion of the feature map will lead to the loss of some semantic information. Sparse convolution completely cuts off the hidden connections between channels, while dense convolution maximizes the retention of these connections. GSConv tries to retain these connections as much as possible, but if it is used in all stages of the model, the network layer of the model will be deeper, increase the resistance to data flow, and significantly increase the inference time. However, when the feature maps reach Neck, they have become slender, the channel dimension reaches the maximum, the width and height dimensions reach the minimum, and no transformation is required. Therefore, using GSConv only in Neck is the best choice. At this stage, using GSConv to process feature maps has less redundant and repeated information, and no compression is required. The MCB module is replaced by a one-time aggregation method to design a cross-level partial network (GSCSP) module VoVGSCSP. While maintaining sufficient accuracy, the use of this module reduces the complexity of calculation and network structure. The VoVGSCSP-DCN-CBAM module combines the deformable convolution DCNv2 module and the CBAM attention mechanism module on its basis. Its structure is as follows: Figure 5 shown.

[0082] Standard convolution units sample input feature maps at fixed positions; pooling layers reduce spatial resolution at a fixed ratio; RoI (region of interest) pooling layers divide RoIs into fixed spatial bins, etc., so there is no internal mechanism to handle geometric transformations. Deformable convolution adds regular grid sampling position offsets and deformable ROI pooling to standard convolution. First, a regular grid R is used to sample the input feature map x; then the sampled values ​​are weighted by w. R can be expressed as:

[0083] R={(-1,-1),(-1,0),...,(0,1),(1,1)}

[0084] After the convolution kernel operation, for each position p0 on the output feature map y (where p0 enumerates the position in R):

[0085] y(p0)=∑w(p0)*x(p0+p n )

[0086] In deformable convolution, the regular grid R adds an offset Δp n ;

[0087] y(p0)=∑w(p0)*x(p0+p n +Δp n )

[0088] Deformable convolution adds an offset to each bin position in the regular bin partition of the previous RoI pooling. The offset is learned from the previous feature map and RoI, enabling adaptive partial localization of objects of different shapes. To further enhance the ability of deformable convolutional networks to manipulate spatial support regions, DCNv2 introduces a modulation mechanism. In which each sample not only undergoes a learned offset, but is also modulated by a learned feature amplitude. With it, the deformable convolutional network module can not only adjust the offset of the perceived input features, but also modulate the input feature amplitude from different spatial position bins. In extreme cases, the module can decide not to perceive signals from a specific position bin by setting its feature amplitude to zero. The impact of the image content from the corresponding spatial position on the module output will be significantly reduced or have no effect. Therefore, the modulation mechanism provides another dimension of freedom for the network module to adjust its spatial support region.

[0089] y(p0)=∑w(p0)*x(p0+p n +Δp k )*Δm k

[0090] where Δp k and Δm k are the learnable offset and modulation scalar of the kth position, respectively. Δp k and Δm k are all obtained through separate convolutional layers applied to the same input feature map x.

[0091] The deep separable convolutional layer (DCNv2) can better capture local details and texture information in the image without being disturbed by insufficient lighting. This helps improve the model's perception of details such as object edges and textures. DCNv2 has strong nonlinear feature extraction capabilities and can adapt to complex changes in low-light images. Therefore, DCNv2 combined with the VoVGSCSP module can better process low-light images and extract more representative features.

[0092] The CBAM attention module can help the model focus on important features in the image and suppress noise and irrelevant information. Its structure is as follows: Figure 6 As shown in the figure. Under low-light conditions, there is often a lot of noise in the image, and the CBAM module can extract key information from low-light images and highlight their features by introducing channel attention mechanisms and spatial attention mechanisms, making the target easier to detect and identify. In addition, the CBAM module can also adaptively adjust the attention weight to adapt to different scales and complexities, thereby improving the robustness and generalization ability of detection.

[0093] The design of the VoVGSCSP-DCN-CBAM module structure fully considers the characteristics and challenges of images under low-light conditions. By introducing technical means such as global sparse convolutional spatial paths, depthwise separable convolutional layers, and CBAM attention modules, the model can process low-light images more effectively, extract more representative features, and focus on important target areas, thereby improving the accuracy of low-light target detection.

[0094] It should be noted that the steps of replacing the convolution layer before the upsampling in the improved Neck network with the coordinated coordinate convolution to construct the low-illumination target detection model include: obtaining the input feature map before the upsampling operation in the improved Neck network; defining each pixel position of the input feature map as a coordinate vector; splicing the converted coordinate vector with the input feature map to obtain the spliced ​​feature map; and performing a convolution operation on the spliced ​​feature map using the convolution kernel of the coordinated coordinate convolution to construct the low-illumination target detection model.

[0095] In the specific implementation, the convolution of the previous layer of upsampling is replaced by the coordinated coordinate convolution CoordConv, whose structure is as follows Figure 7 As shown in Figure 1. Compared with traditional convolution, the structure of coordinated coordinate convolution includes coordinate definition, coordinate transformation, coordinate and feature map splicing, and convolution operation. First, in this structure, each position of the input feature map is defined as a coordinate vector. Then, the horizontal and vertical coordinates of each input feature point are converted into coordinate vectors and spliced ​​with the input feature map. In this way, the coordinate information is integrated into the feature map as an additional channel. Finally, the convolution operation is applied to the spliced ​​feature map. In addition to perceiving local features at each position, the convolution kernel can also access the coordinate information of the position. This design enables coordinated coordinate convolution to better perceive and utilize the spatial position information of the input features, thereby improving the performance and accuracy of the model in position-related tasks. CoordConv only inherits the first two characteristics of traditional convolution, and the running network structure will maintain or discard translation invariance according to the learning situation. It seems that this will damage the inductive ability of the model, but in fact, taking out part of the network tolerance to model non-translation invariance and perceive the spatial position information in the feature map will actually improve the generalization ability of the model.

[0096] It should be noted that the coordinated coordinate convolution is used to replace the previous convolution operation in the convolution layer before the upsampling operation. The coordinated coordinate convolution introduces the coordinate information of the input feature map during the convolution process. By adding coordinate information to the convolution kernel, the coordinated coordinate convolution can help the low-light target detection model better understand the spatial structure and positional relationship in the feature map. In this case, the convolution layer before the upsampling is replaced by the coordinated coordinate convolution to improve the detection accuracy of the low-light target detection model for targets in low-light environments. Therefore, the low-light target detection model in this embodiment is as follows: Figure 8 shown.

[0097] In the specific implementation, by improving the network structure and feature fusion module of the backbone network of the preset YOLOv7-tiny detection model, and replacing the convolution layer before the upsampling operation of the second low-illumination target detection model with a coordinated coordinate convolution, the low-illumination target detection model is constructed. It can effectively improve the detection accuracy of the low-illumination target detection model for targets in low-illumination environments, and increase the complexity and expression ability of the low-illumination target detection model, thereby reducing the risk of overfitting.

[0098] Step S30: training the low-illumination target detection model based on the training set and the validation set to generate a target detection model.

[0099] In a specific implementation, when training the low-light target detection model using a training set, the low-light target detection model will continuously optimize parameters according to the training results to minimize the error of target detection.

[0100] It can be understood that the optimized low-light target detection model is trained based on the training set to obtain the optimal low-light target detection model. The input image size is set to 640*640, the batch size is 16, the number of training iterations is 500, the initial learning rate is 0.01, the learning rate momentum is 0.937, the weight decay coefficient is 0.0005, and the pre-trained weights are not used in the whole process. After the training is completed, the model will save the best weight file best.pt with the best training results.

[0101] During the training process, various model evaluation indicators are used to determine the superiority of the optimized low-light detection network. The indicators for evaluating superiority include: Precision (precision / accuracy), Recal1 (recall / recall), AF (Average Precision). With recall as the horizontal axis and precision as the vertical axis, a PR curve can be drawn. The area under the PR curve is defined as AP, mAP (the average AP of each category). FPS (frames per second). The calculation formulas for each evaluation indicator are as follows:

[0102]

[0103]

[0104]

[0105]

[0106] Among them, TP represents the correct detection box, FP represents the false detection box, FN represents the missed detection box, len(GTs) represents the number of all label boxes, AP represents the detection accuracy of a target, N represents the number of detection categories, Frames represents the number of frames, and Time represents the detection time.

[0107] Step S40: Obtain the image information to be detected, detect the image information to be detected according to the target detection model and obtain the detection result.

[0108] After the low-light target detection model is trained, the low-light target detection model can be used to detect the image to be detected under low light.

[0109] In the specific implementation, the comparison between the detection results of this embodiment and YOLOv7-tiny is shown in Table 1:

[0110]

[0111] Table 1 Comparative experimental results

[0112] In order to verify the effectiveness of the improved modules in this embodiment on the model improvement, under the same parameter conditions, this embodiment conducts ablation experiments on three innovative points. Whether it is the replacement of the ELAN-DBB module, the use of the VoVGSCSP-DCN-CBAM module or the introduction of the coordinated coordinate convolution structure, good experimental results are achieved in the experiments, but the combination of the three is the best. The results of the ablation experiment are shown in Table 2:

[0113]

[0114] Table 2 Ablation experiment

[0115] The effective improvement of the algorithm can be clearly seen from the data in the two tables in the above comparative experiment and ablation experiment.

[0116] This embodiment obtains a preset data set and preprocesses the preset data set to determine a training set, a validation set, and a test set; introduces a preset backbone network part, a preset neck structure, and a coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model; trains the low-light target detection model based on the training set and the validation set to generate a target detection model; obtains image information to be detected, detects the image information to be detected according to the target detection model, and obtains the detection result. The low-light target detection model is used for detection to improve the low-light target detection model's ability to capture target details under low-light conditions, thereby alleviating the problem caused by insufficient lighting. In addition, by improving the preset YOLOv7-tiny detection model structure, the performance and accuracy of target detection in low-light environments are effectively improved.

[0117] In addition, an embodiment of the present application also proposes a computer-readable storage medium, on which is stored a low-light target detection program improved based on YOLOv7-tiny. When the low-light target detection program improved based on YOLOv7-tiny is executed by a processor, the steps of the low-light target detection method improved based on YOLOv7-tiny as described above are implemented.

[0118] Reference Fig. 9 , Fig. 9 This is a structural block diagram of the first embodiment of the low-light target detection system improved based on YOLOv7-tiny in this application.

[0119] like Fig. 9 As shown, the low-light target detection system based on the YOLOv7-tiny improvement proposed in the embodiment of the present application includes:

[0120] A preset data set acquisition module 10 is used to acquire a preset data set and preprocess the preset data set to determine a training set, a validation set, and a test set;

[0121] A low-light target detection model generation module 20 is used to introduce a preset backbone network part, a preset neck structure, and a coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model;

[0122] The target detection model generation module 30 is used to train the low-light target detection model based on the training set and the validation set to generate the target detection model;

[0123] The detection result generating module 40 is used to obtain the image information to be detected, detect the image information to be detected according to the target detection model and obtain the detection result.

[0124] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0125] This embodiment obtains a preset data set and preprocesses the preset data set to determine a training set, a validation set, and a test set; introduces a preset backbone network part, a preset neck structure, and a coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model; trains the low-light target detection model based on the training set and the validation set to generate a target detection model; obtains image information to be detected, detects the image information to be detected according to the target detection model, and obtains the detection result. The low-light target detection model is used for detection to improve the low-light target detection model's ability to capture target details under low-light conditions, thereby alleviating the problem caused by insufficient lighting. In addition, by improving the preset YOLOv7-tiny detection model structure, the performance and accuracy of target detection in low-light environments are effectively improved.

[0126] In one embodiment, the preset data set acquisition module 10 is also used to acquire a preset data set, determine a target low-light image set and a target bounding box in the preset data set; obtain a conversion result after converting the target low-light image set and the target bounding box into the YOLO training format; obtain a preset division ratio, and divide the conversion result according to the preset division ratio to determine a training set, a validation set, and a test set.

[0127] In one embodiment, the target detection model generation module 30 is also used to introduce an efficient layer aggregation network module that integrates multiple branch blocks, and generate a preset backbone network part based on the efficient layer aggregation network module that integrates multiple branch blocks; introduce a cross-level network module of the attention mechanism, and construct a preset neck structure based on the cross-level network module of the attention mechanism, wherein the cross-level network module of the attention mechanism includes a deformable convolution DCNv2 module and a CBAM attention mechanism module; introduce coordinated coordinate convolution, wherein the structure of the coordinated coordinate convolution includes coordinate definition, coordinate transformation, coordinate and feature map splicing and convolution operation.

[0128] In one embodiment, the target detection model generation module 30 is also used to adopt an efficient layer aggregation network module that fuses multi-branch blocks in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks; obtain the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks, and replace the cross-level network module of the attention mechanism to obtain an improved Neck network; replace the convolution layer of the previous layer of upsampling in the improved Neck network with a coordinated coordinate convolution to construct a low-light target detection model.

[0129] In one embodiment, the target detection model generation module 30 is also used to obtain the network structure of the backbone network in the preset YOLOv7-tiny detection model; replace the network structure with an efficient layer aggregation network module that integrates multiple branch blocks, and obtain output feature maps of different layers by integrating the efficient layer aggregation network module that integrates multiple branch blocks during the forward propagation process; link the output feature maps of different layers to generate an improved efficient feature extraction network with multiple branch blocks.

[0130] In one embodiment, the target detection model generation module 30 is also used to obtain neck structure information in an improved efficient feature extraction network with a multi-branch block; replace the preset neck structure with the neck structure information; perform a convolution operation on the acquired input information through a first convolution layer to obtain a first result; use the first result as the input of the second convolution layer, and perform a convolution operation on the first result through the second convolution layer to obtain a second result; pass the first result through a GS bottleneck module and a depth-separable convolution layer respectively to obtain a third result; splice the second result and the third result to obtain a splicing feature; and obtain an improved Neck network based on the splicing feature and the CBAM attention mechanism module.

[0131] In one embodiment, the target detection model generation module 30 is also used to obtain an input feature map before the upsampling operation in the improved Neck network; define each pixel position of the input feature map as a coordinate vector; splice the converted coordinate vector with the input feature map to obtain a spliced ​​feature map; use the convolution kernel of the coordinated coordinate convolution to perform a convolution operation on the spliced ​​feature map to construct a low-illumination target detection model.

[0132] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0133] In addition, for technical details not described in detail in this embodiment, please refer to the method for low-light target detection provided in any embodiment of the present application, and will not be repeated here.

[0134] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0135] The serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. Through the description of the above implementation modes, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware, but in many cases the former is a better implementation mode. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk), including several instructions to enable a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present application. The above is only a preferred embodiment of the present application, and the patent scope of the present application is not limited thereto. All equivalent structures or equivalent process changes made by using the contents of the present application specification and the drawings, or directly or indirectly applied in other related technical fields, are similarly included in the patent protection scope of the present application.

Claims

1. A low-light target detection method based on YOLOv7-tiny, characterized in that: include: Obtain a preset data set, and preprocess the preset data set to determine a training set, a validation set, and a test set; Introduce the preset backbone network part, preset neck structure and coordinated coordinate convolution into the preset YOLOv7-tiny detection model to generate a low-light target detection model; Training the low-light target detection model based on the training set and the validation set to generate a target detection model; Acquire the image information to be detected, detect the image information to be detected according to the target detection model and obtain the detection result.

2. The improved low-light target detection method based on YOLOv7-tiny according to claim 1, characterized in that: The step of obtaining a preset data set and preprocessing the preset data set to determine a training set, a validation set, and a test set includes: Acquire a preset data set, and determine a target low-light image set and a target bounding box in the preset data set; Converting the target low-light image set and the target bounding box into a YOLO training format to obtain a conversion result; A preset division ratio is obtained, and the conversion result is divided according to the preset division ratio to determine a training set, a validation set, and a test set.

3. The improved low-light target detection method based on YOLOv7-tiny according to claim 1, characterized in that: Before the step of introducing a preset backbone network part, a preset neck structure, and a coordinated coordinate convolution into the preset YOLOv7-tiny detection model to generate a low-light target detection model, the method further includes: Introducing an efficient layer aggregation network module integrating multiple branch blocks, and generating a preset backbone network part according to the efficient layer aggregation network module integrating multiple branch blocks; Introducing a cross-level network module of an attention mechanism, and constructing a preset neck structure according to the cross-level network module of the attention mechanism, wherein the cross-level network module of the attention mechanism includes a deformable convolution DCNv2 module and a CBAM attention mechanism module; A coordinated coordinate convolution is introduced, wherein the structure of the coordinated coordinate convolution includes coordinate definition, coordinate transformation, coordinate and feature map concatenation and convolution operation.

4. The improved low-light target detection method based on YOLOv7-tiny according to claim 3, characterized in that: The step of introducing a preset backbone network part, a preset neck structure, and coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model includes: The efficient layer aggregation network module integrating the multi-branch blocks is adopted in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks; Obtaining a feature fusion module of the neck portion corresponding to the improved efficient feature extraction network with multi-branch blocks, and replacing the cross-level network module of the attention mechanism to obtain an improved Neck network; The convolution layer before upsampling in the improved Neck network is replaced by the coordinated coordinate convolution to construct the low-illumination target detection model.

5. The improved low-light target detection method based on YOLOv7-tiny according to claim 4, characterized in that: The step of using the efficient layer aggregation network module integrating the multi-branch block in the preset YOLOv7-tiny detection model to generate an improved efficient feature extraction network with multi-branch blocks includes: Get the network structure of the backbone network in the preset YOLOv7-tiny detection model; The network structure is replaced by the efficient layer aggregation network module of the fused multi-branch block, so as to obtain output feature maps of different layers through the efficient layer aggregation network module of the fused multi-branch block during the forward propagation process; Get the current replaced network structure and generate an efficient feature extraction network with multi-branch blocks.

6. The improved low-light target detection method based on YOLOv7-tiny according to claim 4, characterized in that: The step of obtaining the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks, and replacing the cross-level network module of the attention mechanism to obtain the improved Neck network includes: Obtaining a feature fusion module for the neck portion corresponding to the improved efficient feature extraction network with multi-branch blocks; Replace the cross-level network module of the attention mechanism with the feature fusion module of the neck part corresponding to the improved efficient feature extraction network with multi-branch blocks; Performing a convolution operation on the acquired input information through the first convolution layer to obtain a first result; Using the first result as input of a second convolutional layer, and performing a convolution operation on the first result through the second convolutional layer to obtain a second result; Pass the first result through the GS bottleneck module and the depth-separable convolutional layer respectively to obtain a third result; splicing the second result and the third result to obtain a splicing feature; An improved Neck network is obtained based on the splicing features and the CBAM attention mechanism module.

7. The improved low-light target detection method based on YOLOv7-tiny according to claim 4, characterized in that: The step of replacing the convolution layer of the previous upsampling layer in the improved Neck network with the coordinated coordinate convolution to construct the low-illumination target detection model includes: Before the upsampling operation in the improved Neck network, obtaining an input feature map; Defining each pixel position of the input feature map as a coordinate vector; Splicing the converted coordinate vector with the input feature map to obtain a spliced ​​feature map; The convolution operation is performed on the spliced ​​feature map using the convolution kernel of the coordinated coordinate convolution to construct the low-illumination target detection model.

8. A low-light target detection system based on YOLOv7-tiny, characterized in that: include: A preset data set acquisition module is used to acquire a preset data set and preprocess the preset data set to determine a training set, a validation set, and a test set; A low-light target detection model generation module is used to introduce a preset backbone network part, a preset neck structure, and coordinated coordinate convolution into a preset YOLOv7-tiny detection model to generate a low-light target detection model; A target detection model generation module, used to train the low-light target detection model based on the training set and the verification set to generate a target detection model; The detection result generation module is used to obtain the image information to be detected, detect the image information to be detected according to the target detection model and obtain the detection result.

9. A computer device, characterized in that: The device comprises: a memory and a processor, and when the processor runs the computer instructions stored in the memory, the processor executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 7.

Images