A method for object detection in complex background based on bilateral attention mechanism

Abstract

The present invention discloses a method for detecting targets in complex backgrounds based on a bilateral attention mechanism, which can be used for accurate foreground target detection in complex backgrounds. The invention mainly includes: constructing a training set, a validation set and a test set based on a disclosed target detection data set in complex backgrounds; constructing an artificial neural network detection model Bi‑SINet based on a bilateral attention mechanism; optimizing the Bi‑SINet model using an SGD optimizer on a Pytorch deep learning platform; and evaluating the detection performance of the converged Bi‑SINet network model on the constructed test set. The present invention can achieve better detection performance than the current main target detection algorithm SINet in complex backgrounds. While reducing the mean absolute error, the present invention achieves a higher enhancement-alignment index, a structural index and a weighted F index, and is a more accurate target detection algorithm in complex backgrounds.

Description

Technical Field

[0001] The present invention relates to a method for object detection under complex backgrounds based on a bilateral attention mechanism, and is applicable to the technical field of object detection under complex backgrounds in computer vision. Background Art

[0002] Images and videos are important sources of information for humans, so the use of computers to analyze and process massive images and videos has also been vigorously developed. As one of the basic tasks in the field of computer vision, object detection is an important means to help computers understand image data. It has broad application prospects in the fields of pedestrian detection, vehicle detection, autonomous driving, security systems, and medical treatment.

[0003] Thanks to the development of deep learning technology, target detection has attracted widespread attention in recent years and has achieved great success. A large number of efficient detection algorithms have been proposed. Existing target detection algorithms based on convolutional neural networks (CNN) are mainly divided into two categories: single-stage detection algorithms and two-stage detection algorithms. Representative algorithms of single-stage detection algorithms are YOLO and SSD, and representative algorithms of two-stage algorithms are Faster R-CNN and Cascade R-CNN. Current general target detection has made great progress and can achieve a high detection accuracy, but due to the complexity of real scenes, as well as the camouflage color and occlusion of targets, high-precision detection has brought great challenges. In some application fields such as medical, agricultural, marine, and military, the background complexity of image data collected is often high. Due to factors such as light intensity, color complexity, and the camouflage color of the object itself, the target to be detected is often integrated with the surrounding environment, the gap between the foreground and the background is extremely small, and the edge of the object is difficult to distinguish, resulting in the inability of general target detection algorithms to be directly applied. Therefore, it is of great significance to carry out research on target detection algorithms under complex backgrounds. Summary of the invention

[0004] Aiming at the problem of insufficient detection accuracy of existing target detection algorithms under complex backgrounds, the present invention proposes a target detection method under complex backgrounds based on the bilateral attention mechanism on the basis of the current algorithm SINet, which is named Bi-SINet. The Bi-SINet network model in the present invention adopts a progressive detection framework to perform reasoning detection on the target to be detected from coarse to fine. At the same time, bilateral attention is used to improve the efficiency of feature fusion. The detection performance of the trained Bi-SINet network model is better than that of the Bi-SINet model. While reducing the mean absolute error (MAE), the enhanced-alignment measure (EM), structure measure (SM) and weighted F measure (WFM) of the detection results are improved, indicating that the present invention effectively improves the accuracy of target detection under complex backgrounds.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for object detection in a complex background based on a bilateral attention mechanism, characterized by comprising the following steps:

[0007] Step S1: construct a training set, a validation set, and a test set based on a publicly available complex background object detection dataset;

[0008] Step S2: construct a Bi-SINet network model based on the bilateral attention mechanism, and use the constructed training set to supervise the Bi-SINet network model until the model converges to the optimal performance;

[0009] Step S3: Input the image to be detected into the converged Bi-SINet network model obtained in step S2 for forward operation to obtain the detection result. The converged Bi-SINet network model is tested on the constructed test set, and the model performance is evaluated according to the test results.

[0010] Furthermore, the step S1 specifically includes:

[0011] Step S101: Obtain a public complex background target detection dataset, including the COD10K dataset, the CAMO dataset, and the CHAMELEON dataset;

[0012] Step S102: The constructed training set contains 4040 pairs of image label pairs, the constructed validation set contains 101 pairs of image label pairs, and the constructed test set contains 2352 pairs of image label pairs.

[0013] Furthermore, the step S2 specifically includes:

[0014] Step S203: In the training phase, the cross entropy loss function and the intersection-over-union loss function are used to supervise the five detection results output by the model. The overall loss function can be expressed as L overall To express it, the calculation formula is:

[0015]

[0016] in, represents the structurally weighted cross entropy loss between the k-th layer detection result and the true label, and It represents the structural weighted intersection-over-union loss between the k-th layer detection result and the true label. Their calculation expressions are:

[0017]

[0018]

[0019] Where H represents the image height, W represents the image width, and mask GT (x, y) represents the predicted result of the kth layer and the value of the position coordinate (x, y) in the true label, γ is a parameter related to the structural weight, set to γ ​​= 5, w(x, y) represents the structural weight corresponding to the position with coordinates (x, y), and its expression is as follows:

[0020]

[0021] Among them, A xy Represents the set of surrounding pixels centered on the pixel with coordinates (x, y);

[0022] Step S204: During the training process, the network parameters are iteratively updated using the stochastic gradient descent optimization algorithm, and the learning rate is decayed using the polynomial method. During the training process, the model performance is verified using the constructed validation set every 200 iterations to ensure that the model converges to the optimal performance;

[0023] Step S205: using the Bi-SINet network model with fixed parameters after optimization convergence to perform forward calculation on the input image to be detected, a foreground target segmentation map can be obtained as a detection result.

[0024] Furthermore, the step S3 specifically includes:

[0025] Step S301: After normalizing the image to be detected, input it into the converged Bi-SINet network model for detection. Among the detection results of 5 different scales output, select the one with the largest scale. As the final detection result mask of the model pred .

[0026] Step S302: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the mean absolute error MAE, the calculation formula is as follows:

[0027]

[0028] Among them, H represents the image height, W represents the image width, and mask pred (x,y) and mask GT (x, y) represents the predicted result and the value of the position coordinate (x, y) in the true label respectively;

[0029] Step S303: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the objective evaluation index EM, the calculation formula is as follows:

[0030]

[0031] Among them, H represents the image height, W represents the image width, φ FM represents the enhanced alignment matrix;

[0032] Step S304: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the objective evaluation index SM, the calculation formula is as follows:

[0033] SM=αS0+(1-α)S r (7)

[0034] Where S0 represents the structural similarity to the target, S r It indicates the structural similarity of the region, and the weight a that balances the two is usually 0.5;

[0035] Step S305: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the objective evaluation index WFM, the calculation formula is as follows:

[0036]

[0037] Where P w Represents the weighted corrected accuracy, R w represents the weighted corrected recall rate, β is the factor that balances precision and recall rate, and is set to 0.3;

[0038] Finally, the converged Bi-SINet network model can be used to detect targets in complex backgrounds, input the image to be detected for forward calculation, and output the predicted foreground target segmentation map as the detection result.

[0039] The beneficial effects of the present invention are as follows: the Bi-SINet network model proposed in the present invention adopts a bilateral attention mechanism to simulate the signal processing mechanism of the human brain, screens and strengthens the target features, and thus improves the efficiency of feature fusion. The multi-layer detection results output by the progressive detection framework enhance the supervision and constraint capabilities of the features of each layer of the model during the training phase, and can detect targets in complex backgrounds from coarse to fine during detection. Compared with the current main target detection models under complex backgrounds, the Bi-SINet network model proposed in the present invention can achieve more accurate detection results, while effectively alleviating the phenomenon of false detection and missed detection, which is specifically reflected in: the mean absolute error MAE of the detection results is significantly reduced, and the structural index SM, the enhancement-alignment index EM and the weighted F index WFM are also significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flowchart of the target detection method in a complex background based on the bilateral attention mechanism in Example 1.

[0041] Figure 2 This is the Bi-SINet network model structure diagram in Example 1.

[0042] Figure 3 This is a structural diagram of the feature fusion module based on bilateral attention in Example 1.

[0043] Figure 4 This is the internal structure diagram of the bilateral attention mechanism in Example 1.

[0044] Figure 5 This is a flowchart of Bi-SINet network training and prediction in Example 1.

[0045] Figure 6 The comparison of the detection performance of the method of the present invention and the current main methods in Example 1 on the evaluation indicators.

[0046] Figure 7 The comparison of the detection performance of the method of the present invention and the current main method in terms of image effect in Example 1 is shown. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Example 1

[0049] See also Figure 1-Figure 5 ,This implementation provides a method for object detection in complex ,backgrounds based on bilateral attention mechanism.

[0050] Specifically, see Figure 1 , this method specifically includes:

[0051] Step S1: Obtain public target detection datasets under complex backgrounds, including: COD10K dataset, CAMO dataset, and CHAMELEON dataset, and construct training sets, validation sets, and test sets based on them;

[0052] More specifically, the constructed training set contains 3040 pairs of image data pairs in the COD10K dataset and 100 pairs of image data pairs in the CAMO dataset, totaling 4040 data items; the constructed validation set contains 101 pairs of image data pairs in the COD10K dataset; the constructed test set contains 2026 pairs of image data pairs in the COD10K dataset, 250 pairs of image data pairs in the CAMO dataset, and 76 pairs of image data pairs in the CHAMELEON dataset, totaling 2352 data items.

[0053] Step S2: Construct a Bi-SINet network model based on the bilateral attention mechanism.

[0054] More specifically, the specific structure of the constructed Bi-SINet network model is as follows Figure 2 As shown in the figure, the Bi-SINet network model is mainly composed of a feature extractor and a progressive detection framework. The feature extractor uses the ResNet-50 network, and the feature extractor outputs 4 layers of features of different scales to the progressive detection framework for detection. Figure 2 As shown in the figure, the progressive detection framework mainly consists of a receptive field enhancement module, a bilateral attention-based feature fusion module, and a detection module.

[0055] a) The receptive field enhancement module contains four convolution branches with different receptive field scales and one residual connection branch. Each receptive field branch uses two convolution layers and one dilated convolution layer to simulate receptive fields of different scales. The features enhanced by different receptive field branches are fused by splicing and convolution.

[0056] b) The feature fusion module based on bilateral attention contains a bilateral attention mechanism to assist in the fusion of high-level features with the current layer features. Its main structure is as follows Figure 3 As shown, for the feature fusion of the kth layer (k = 1, 2, 3), the high-level enhanced feature f k ' +1 First, the feature is enhanced by the bilateral attention module, and then combined with the high-level detection result pred k+1 The enhanced foreground target feature F is obtained by upsampling by 2 and multiplying them together. After being concatenated with the feature of this layer in the channel dimension, the feature fusion result f of this layer is output through a convolution layer. k ″.

[0057] c) The main structure of the detection module is a small Unet network, which only performs downsampling and upsampling once in the encoding stage and decoding stage respectively.

[0058] More specifically, the internal structure of the bilateral attention mechanism is as follows Figure 4 As shown in the figure, its main components are the value attention and distance attention branches. The input features are encoded on the two attention branches respectively, and the encoded features are output after feature concatenation and convolution operations. Among them, when calculating the value attention encoding of a certain pixel feature, it is necessary to comprehensively consider the correlation between the current pixel feature and the surrounding pixel feature in terms of value, and use the dot product operation of the feature to measure it. Its calculation expression is as follows:

[0059]

[0060] in, represents the value of pixel i, x i represents the features corresponding to the surrounding pixels, v represents the set of pixels within a 3×3 range centered on pixel i, f(x,y) represents the dot product operation between two pixels, and C(x) represents normalization. Therefore, the corresponding dot product result is subjected to softmax operation in the j dimension.

[0061] When calculating the distance attention encoding of a pixel feature, it is necessary to consider the correlation between the current pixel as the center and the distance between the surrounding pixel features and the current pixel feature, and use the Euclidean distance and Gaussian distribution function to characterize it. The distance attention encoding process can be expressed as follows:

[0062]

[0063] in, represents the distance attention feature encoding result of the current pixel i, p i Represents the position coordinates of pixel i, p jIndicates the position coordinates of the surrounding pixels. j represents the input features of the surrounding pixels, s represents the set of pixels in the 5×5 range around the current pixel i, and C(x) represents the normalization factor, corresponding to the number of elements in the set s. is a Gaussian function, and its expression is:

[0064]

[0065] Among them, d represents the Euclidean distance between two pixels, α s is a smoothing parameter, set to a in the model s =1.5.

[0066] Step S3: On the Pytorch deep learning framework, the Bi-SINet network model is supervised and trained using the constructed training set, and the model performance is verified using the validation set at regular intervals until the model converges to the optimal performance;

[0067] More specifically, the training and prediction process of the Bi-SINet network model is as follows Figure 3 As shown, including:

[0068] Step S301: preprocess the training data. First, the size of the original image and the corresponding true label is modified to 384×384 using bilinear interpolation. Then, image data enhancement processing such as random scaling and random rotation is performed. Finally, the image is normalized and input into the Bi-SINet network model for training.

[0069] Step S302: The Bi-SINet network model extracts features from the input image, and then inputs the extracted multi-layer features into the progressive detection framework for detection. The progressive detection framework performs foreground target segmentation based on each layer of features, and outputs a total of 5 different scales of detection results for multi-layer supervision training. To represent the 5 detection results output by the Bi-SINet network model;

[0070] Step S303: In the training phase, the cross entropy loss function and the intersection-over-union loss function are used to supervise the 5 detection results output by the model. The overall loss function can be expressed as L overall To express it, the calculation formula is:

[0071]

[0072] in, represents the structurally weighted cross entropy loss between the k-th layer detection result and the true label, and It represents the structural weighted intersection-over-union loss between the k-th layer detection result and the true label. Their calculation expressions are:

[0073]

[0074]

[0075] Where H represents the image height, W represents the image width, and mask GT (x, y) represents the predicted result of the kth layer and the value of the position coordinate (x, y) in the true label, γ is a parameter related to the structural weight, set to γ ​​= 5, w(x, y) represents the structural weight corresponding to the position with coordinates (x, y), and its expression is as follows:

[0076]

[0077] Among them, A xy Represents the set of surrounding pixels centered on the pixel with coordinates (x, y);

[0078] Step S304: During the training process, the SGD optimizer is used to iteratively update the network parameters, and the learning rate is decayed by the polynomial method. During the training process, the model performance is verified using the constructed validation set at intervals of a certain number of iterations to ensure that the model converges to the optimal performance;

[0079] Step S305: using the Bi-SINet network model with fixed parameters after convergence to perform forward calculation on the input image to be detected, a foreground target segmentation map can be obtained as a detection result.

[0080] It should be noted that the Bi-SINet network model provided in this embodiment is not limited to the Pytorch deep learning framework. As long as the constructed training data set can be trained and convergence is achieved after several iterations during the training process, target detection in complex backgrounds can be finally achieved.

[0081] Step S4: Input the normalized image to be detected into the converged Bi-SINet network model obtained in step S3 for forward operation to obtain the detection result. The converged Bi-SINet network model is tested on the constructed test set, and the model performance is evaluated according to the detection result.

[0082] More specifically, after the image to be detected is normalized, it is input into the converged Bi-SINet network model for detection. Among the detection results of 5 different scales output, the one with the largest scale is selected. As the final detection result mask of the model predThe formulas for calculating the four evaluation indicators for evaluating model performance are as follows:

[0083] 1) Mask based on the detection results of the model pred And the corresponding true label mask GT , calculate the mean absolute error MAE, the calculation formula is as follows:

[0084]

[0085] Among them, H represents the image height, W represents the image width, and mask pred (x,y) and mask GT (x, y) represents the predicted result and the value of the position coordinate (x, y) in the true label respectively;

[0086] 2) Mask based on the detection results of the model pred And the corresponding true label mask GT , calculate the objective evaluation index EM, the calculation formula is as follows:

[0087]

[0088] Among them, H represents the image height, W represents the image width, φ FM represents the enhanced alignment matrix;

[0089] 3) Mask based on the detection results of the model pred And the corresponding true label mask GT , calculate the objective evaluation index SM, the calculation formula is as follows:

[0090] SM=αS0+(1-α)S r (7)

[0091] Where S0 represents the structural similarity to the target, S r It indicates the structural similarity of the region, and the weight a that balances the two is usually taken as 0.5;

[0092] 4) Mask based on the detection results of the model pred And the corresponding true label mask GT , calculate the objective evaluation index WFM, the calculation formula is as follows:

[0093]

[0094] Where P w Represents the weighted corrected accuracy, R w represents the weighted corrected recall rate, β is the factor that balances precision and recall, and is usually set to 0.3;

[0095] It should be noted that the smaller the mean absolute error M, the better the structural index S, the enhancement-alignment index E and the weighted F index. The larger the value, the more accurate the output detection result. In addition, there are more than the above four evaluation indicators to measure the target detection effect, as long as they can show the similarity or distinction between the predicted foreground target segmentation map and the true label.

[0096] Figure 6 The comparison of the target detection performance of the proposed method and the SINet network model on the constructed test set is shown. As can be seen from the figure, the Bi-SINet network model proposed in the present invention reduces the mean absolute error compared to the SINet model, while the SM, EM and WFM indicators are all improved, indicating that the present invention can effectively improve the accuracy of target detection in complex backgrounds and alleviate the phenomenon of false detection and missed detection.

[0097] Figure 7 This is a comparison of the detection performance of the proposed method and the SINet network model in terms of image effects. From the comparison of the first column of pictures, it can be seen that the Bi-SINet network model can ensure the integrity of the detection results in complex backgrounds, thanks to the progressive detection framework and the coarse-to-fine detection process. From the second and third columns of pictures, it can be seen that the Bi-SINet network model can accurately distinguish the target to be detected from the background target even when the edge area of ​​the foreground target is blurred, effectively improving the accuracy of the detection results. Figure 7 From the comparison of the fourth column of pictures, it can be seen that the Bi-SINet network model proposed in the present invention can effectively avoid the interference caused by complex background and accurately detect the target of interest.

[0098] The matters not described in detail in the present invention are all known technologies to those skilled in the art.

[0099] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for object detection in complex background based on bilateral attention mechanism, characterized in that: The method comprises the following steps: Step S1: construct a training set, a validation set, and a test set based on a publicly available complex background object detection dataset; Step S2: construct a Bi-SINet network model based on the bilateral attention mechanism, and use the constructed training set to supervise the Bi-SINet network model until the model converges to the optimal performance; Step S3: Test the converged Bi-SINet network model on the constructed test set and evaluate the model performance based on the test results; The target detection data set in step S1 includes the data set COD10K, the data set CAMO and the data set CHAMELEON; The step S2 specifically includes: Step S201: preprocessing the training data, firstly resizing the original image to 384×384 using bilinear interpolation, and adjusting the corresponding true label, then performing image data enhancement processing, and finally normalizing the image and inputting it into the Bi-SINet network model for training; Step S202: The Bi-SINet network model extracts features from the input image, and then inputs the extracted multi-layer features into a progressive detection framework for detection; the progressive detection framework performs foreground target segmentation based on each layer of features, and outputs N detection results of different scales. To express; Step S203: In the training phase, the cross entropy loss function and the intersection-over-union loss function are used to supervise the N detection results output by the model. The overall loss function can be expressed as L overall To express it, the calculation formula is: in, represents the structurally weighted cross entropy loss between the k-th layer detection result and the true label, and It represents the structural weighted intersection-over-union loss between the k-th layer detection result and the true label. Their calculation expressions are: Where H represents the image height, W represents the image width, and mask GT (x, y) represents the predicted result of the kth layer and the value of the position coordinate (x, y) in the true label, γ is a parameter related to the structural weight, w(x, y) represents the structural weight corresponding to the position with coordinates (x, y), and its expression is as follows: Among them, A xy Represents the set of pixels around the pixel with coordinates (x, y); Step S204: During the training process, the network parameters are iteratively updated using the stochastic gradient descent optimization algorithm, and the learning rate is decayed using the polynomial method; during the training process, the model performance is verified using the constructed validation set at intervals of several iterations to ensure that the model converges to the optimal performance; Step S205: using the Bi-SINet network model with fixed parameters after optimization convergence to perform forward calculation on the input image to be detected, a foreground target segmentation map can be obtained as a detection result.

2. According to claim 1, a method for object detection in complex background based on bilateral attention mechanism, characterized in that: The step S3 specifically includes: Step S301: After normalizing the image to be detected, input it into the converged Bi-SINet network model for detection. Among the N detection results of different scales output, select the one with the largest scale. As the final detection result mask of the model pred ; Step S302: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the mean absolute error MAE, the calculation formula is as follows: Among them, H represents the image height, W represents the image width, and mask pred (x,y) and mask GT (x, y) represents the predicted result and the value of the position coordinate (x, y) in the true label respectively; Step S303: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the objective evaluation index EM, the calculation formula is as follows: Among them, H represents the image height, W represents the image width, φ FM represents the enhanced alignment matrix; Step S304: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the objective evaluation index SM, the calculation formula is as follows: SM=αS0+(1-α)S r (7) Where S0 represents the structural similarity to the target, S r It indicates the structural similarity of the region, and the weight a that balances the two is usually 0.5; Step S305: Mask based on the detection result of the model pred And the corresponding true label mask GT , calculate the objective evaluation index WFM, the calculation formula is as follows: Where P w Represents the weighted corrected accuracy, R w represents the weighted corrected recall rate, and β is the factor that balances precision and recall.

Images