A target detection algorithm based on infrared and visible light images

Abstract

The application provides a target detection algorithm based on infrared and visible light images, which effectively realizes performance complementation of visible light images and infrared images, reduces noise interference such as light, rain and fog, and greatly improves generalization ability and stability of target detection technology. The algorithm combines deep separable convolution with residual structure, constructs parallel high-efficiency feature extraction networks, extracts infrared and visible light image target information respectively, introduces an adaptive feature fusion module to fuse features of corresponding scales of two branches in a self-learning manner, so that the two types of image information are complementary, and finally, the feature pyramid structure is used to layer by layer fuse deep features with shallow features, so as to improve the detection accuracy of the network to targets of different scales. The application is used in the field of image target recognition and detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image target recognition detection, and particularly relates to a target detection algorithm based on infrared and visible light images. BACKGROUND

[0002] Image target recognition detection technology, as an important branch in the field of digital image processing and pattern recognition, is widely used in automatic driving, medical images, industrial detection, intelligent robots, intelligent video monitoring and many other scenes. Target detection is to locate and extract the target region of interest in the video or image by analyzing the target feature information, and accurately identify the target category and the corresponding bounding box of each region. At present, most target detection algorithms are mainly based on visible light images for recognition and detection. Although visible light images contain rich texture and detail information, the environment of each target in the actual scene is complex, resulting in target occlusion, large scale variation range, uneven illumination and noise interference, etc., which makes the landing of visual detection technology still face great challenges. Infrared images mainly use thermal radiation energy imaging and are less affected by light, but the image contrast is low and the target texture structure and other features are severely lost, which greatly limits its application in target detection field. In order to alleviate the drawbacks of traditional algorithms, researchers apply deep learning methods to target detection, which greatly improves the target detection accuracy through end-to-end training. The specific methods are as follows: 1. Using a dual-channel deep residual convolutional network to extract features from visible light and infrared images respectively, and combining attention mechanism to realize target detection in the region of interest by calculating the Euclidean distance of the features; 2. Through coarse positioning of the target in the two kinds of images, then combining the convolutional network to make decision fusion on the positioning frame to realize efficient detection; 3. Using the difference maximum loss function to guide the convolutional network of the infrared and visible light branches to extract target features, and designing feature enhancement and cascaded semantic expansion modules to improve the detection of targets of different scales; 4. Using the improved YOLOv3 network to detect visible light and infrared images respectively, and then weighting fusion to improve the target detection effect. It can be seen that the detection method based on deep learning and infrared and visible light fusion can effectively improve the target detection effect, but most methods extract features separately and then fuse detection, which fails to fully utilize the target features in the two types of images for information complementation.

[0003] Therefore, studying a target detection method based on the combination of visible light and infrared images can effectively realize performance complementation, reduce noise interference such as light, rain and fog, and has a great promoting effect on the development of target detection technology and the application of actual scenes. SUMMARY

[0004] The technical problem to be solved by the present application is to overcome the shortcomings of the prior art, and provide a target detection algorithm based on infrared and visible light images, which effectively realizes the performance complementarity of visible light images and infrared images, reduces noise interference such as light, rain and fog, and greatly improves the generalization ability and stability of target detection technology.

[0005] The technical solution adopted by the present application is: comprising combining deep separable convolution with residual structure, constructing a high-efficiency parallel feature extraction network, the feature extraction network is composed of three parts of a feature extraction module, a feature fusion module and a detection module, wherein the feature extraction module is composed of two parallel same structure branches for infrared and visible light images, adopts deep separable convolution as a feature extraction basic unit, combines LeakyReLU activation layer, maximum pooling layer and up-sampling operation, and efficiently extracts the feature information of infrared and visible light images from shallow to deep; the feature fusion module models the features of the two branch pooling layers through linear combination, realizes information sharing of infrared and visible light by using autonomous learning, and makes the features extracted by the two branches complementary; the detection module uses multiple deep features of different scales to construct a feature pyramid prediction structure by layer-by-layer up-sampling fusion operation, and improves the detection accuracy of the network for targets of different scales.

[0006] Further, the feature extraction module is composed of an init module and a plurality of stage modules, the init module performs feature preprocessing on the original image, adopts parallel convolution and pooling with a step of 2 to extract target significant features, reduces the image dimension while filtering part of the noise, and guarantees the in-depth extraction of the subsequent structure on the features; the stage module is constructed in a residual structure manner using convolution layers and activation layers, and different stages are reduced in dimension through 2x2 pooling operation with a step of 2.

[0007] Still further, the stage module introduces a residual structure to avoid the problems of gradient vanishing and gradient explosion during training caused by too deep network layers, and uses LeakyReLU as an activation function to reduce the probability of neuron "necrosis", so that the network converges faster,

[0008]

[0009] In the formula, alpha represents an offset, which is a small hyperparameter value, and the default setting is 0.02.

[0010] Still further, the feature fusion module considers the network running efficiency, fuses the last layer of each scale in the feature extraction process, that is, fuses the pooling layer between the last layer of the init module and the stage module, and simultaneously uses the linear weighting method of autonomous learning to replace the direct addition of feature information, so as to avoid introducing too much noise during fusion.

[0011] Further, the fusion calculation formula is as follows

[0012]

[0013] The linear weighted calculation formula (1) and (2) of autonomous learning are as follows

[0014]

[0015] wherein, is the pixel value of the coordinate position (i, j) on the visible light feature map to be fused, and similarly, is the pixel value of the coordinate position (i, j) on the infrared feature map to be fused, and AA , α AB , α BA , α BB is the weight to be learned, y is the output of the corresponding feature map after fusion, and represents updating the weight by error back propagation, and it can be seen from the linear weighted calculation formula (2) of autonomous learning that when α AA = α BB = 1, α AB = α BA = 0, it indicates that the feature information is not fused, and the fusion degree of infrared and visible light image feature information is determined by autonomous learning, thereby achieving optimal combination.

[0016] Further, the detection module takes the output of the feature fusion module as input, adopts concat operation to splice the two-way features of the init module and the stage module as detection input, simultaneously adopts multiple fusion features of different dimensions, and constructs a feature pyramid detection structure in a top-down manner.

[0017] Finally, the pyramid detection structure includes the following steps,

[0018] Step one: adjusting the feature channel to be consistent with the shallow layer feature by point convolution for the fused deep layer feature;

[0019] Step two: up-sampling to the size of the shallow layer feature and splicing with the shallow layer feature;

[0020] Step three: performing convolution operation on the spliced feature information to fully fuse the deep layer feature information;

[0021] Step four: repeating the above up-sampling splicing fusion operation for multiple fusion features of different dimensions in sequence, so that the detection module can fully obtain global and local feature information;

[0022] Step five: the target class and position are predicted by using four scales of features respectively, and the prediction results are screened by a maximum suppression algorithm to obtain the optimal target detection frame, so as to realize accurate detection of each target.

[0023] Compared with the prior art, the present application has the beneficial effects that: the present application proposes an infrared and visible light image fusion target detection method aiming at the shortcomings of the existing visible light based target detection algorithm. The algorithm combines deep separable convolution with residual structure to construct a parallel high-efficiency feature extraction network, extracts infrared and visible light image target information respectively; at the same time, an adaptive feature fusion module is introduced to fuse the features of corresponding scales of the two branches in a self-learning manner, so that the two types of image information are complementary; finally, the deep features are fused layer by layer with the shallow features by using the feature pyramid structure, so as to improve the detection accuracy of the network for targets of different scales. The network of the present application can fully fuse the effective information in infrared and visible light images, and realize target recognition and positioning under the premise of ensuring accuracy and efficiency; at the same time, in the actual substation equipment detection scene, the network also shows good robustness and generalization ability, and can efficiently complete the detection task. Therefore, the present application effectively realizes the performance complementation of visible light images and infrared images, reduces the noise interference such as light, rain and fog, and greatly improves the generalization ability and stability of the target detection technology. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a structural diagram of the feature extraction network of the present application;

[0025] Figure 2 is a schematic diagram of the init module;

[0026] Figure 3 is a schematic diagram of the stage module;

[0027] Figure 4 is a schematic diagram of the feature pyramid detection module;

[0028] Figure 5 is a comparison diagram of single branch and fusion branch target detection results;

[0029] Figure 6 is a comparison diagram of the detection effect of the present application and infrared and visible light network;

[0030] Figure 7 is a comparison diagram of substation experimental detection effect;

[0031] Figure 8 is a schematic diagram of the feature cross fusion module. DETAILED DESCRIPTION

[0032] As Figure 1As shown, the present application includes combining deep separable convolution with residual structure, constructing and arranging high-efficiency feature extraction network, which is composed of feature extraction module, feature fusion module and detection module, wherein the feature extraction module is composed of two parallel branches with the same structure for infrared and visible light images, mainly using deep separable convolution as the basic unit of feature extraction, combining LeakyReLU activation layer, maximum pooling layer, up-sampling and other operations, efficiently extracting shallow to deep feature information of infrared and visible light images; the feature fusion module models the two branch pooling layer features through linear combination, realizes infrared and visible light information sharing by using autonomous learning method, and makes the features extracted by the two branches complementary; the detection module uses multiple deep features of different scales to construct feature pyramid prediction structure by layer-by-layer up-sampling fusion operation, and improves the detection accuracy of the network for targets of different scales.

[0033] Among them, feature extraction is the primary task of target detection, and the quality of the extracted features directly determines the pros and cons of the target detection model. For traditional target detection, features are mainly designed by humans, such as SIFT, HOG, Haar, DPM, etc., by extracting corresponding features in the sliding window and using machine learning for classification to realize target detection. The target detection method based on deep learning expands the scope of feature extraction, and uses end-to-end training learning method to automatically learn target features, avoiding the limitations of manually designed features. Therefore, the detection algorithm based on deep learning can usually obtain better detection effect than traditional methods. Based on this, the present application uses the current classic deep learning network to design a parallel double-branch feature extraction network suitable for infrared and visible light images.

[0034] To effectively extract the shallow and deep features of each target in the image, the feature extraction structure constructed by the present application uses multiple sub-modules with different feature scales in series to form a stack, and the detailed structure is shown in the following table.

[0035]

[0036] The feature extraction module is composed of an init module and multiple stage modules, the init module (such as Figure 2 ) mainly performs feature preprocessing on the original image, uses parallel convolution and pooling with a step size of 2 to extract target salient features, reduces the image dimension while filtering part of the noise, and ensures the in-depth extraction of features by the subsequent structure; the stage module (such as Figure 3)Mainly uses convolution layer, activation layer to construct in residual structure way, and the dimensionality reduction is carried out between different stages through 2x2 pooling operation with 2 step lengths.Because the feature extraction needs to be carried out to infrared and visible light images respectively, in order to avoid that the network calculation amount is too large, stage module adopts deep separable convolution to replace traditional convolution to extract features, and the network parameter and calculation amount are effectively reduced.Although the feature information extracted by deep separable convolution is reduced compared with traditional convolution, the information fusion of double branches can better make up for the feature loss and other problems.

[0037] The stage module introduces residual structure to avoid the problems such as gradient vanishing and gradient explosion during training caused by too deep network layer, and uses LeakyReLU as an activation function, reduces the probability of neuron necrosis, and makes the network converge faster,

[0038]

[0039] In the formula, alpha represents an offset, which is a hyperparameter with a small value, and the default setting is 0.02, and as can be seen from the LeakyReLU formula, when the input is less than zero, the output of the function is not zero, and the derivative is a fixed value after derivation, thereby avoiding the problem that the neurons in the negative interval do not learn.

[0040] Generally, the visible light image contains rich color, texture and other information, and can provide more detailed information, but is easily affected by light intensity, weather and the like; and the infrared image uses the thermal radiation energy size of the target to image, and is not affected by light, but the image contrast is low, and part of the appearance features such as texture and structure of the target are lost. Therefore, by fusing the infrared and visible light image information, the target can be better enhanced and discovered. Based on this, the application designs a feature cross fusion module based on the feature extraction structure, so that the extracted infrared and visible light image information is complementary, and the fusion structure is as shown in Figure 8

[0041] The feature fusion module considers the network operation efficiency, fuses the last layer of each scale in the feature extraction process, that is, the pooling layer between the last layer of the init module and the stage module (the init layer is not drawn, and i takes values 1, 2, 3 and 4), and adopts a self-learning linear weighting method to replace direct addition of feature information, to avoid introducing too much noise during fusion. The specific fusion calculation formula is as follows:

[0042]

[0043] The self-learning linear weighting calculation formulas (1) and (2) are as follows

[0044] ​

[0045] wherein, is the pixel value of the coordinate position (i, j) on the visible light feature map to be fused, and similarly, is the pixel value of the coordinate position (i, j) on the infrared feature map to be fused, and AA ,α AB ,α BA ,α BB is the weight to be learned, y is the output of the corresponding feature map after fusion, and represents updating the weight by error back propagation, and as can be seen from the linear weighting calculation formula (2) learned autonomously, when AA = α BB = 1, and AB = α BA = 0, it indicates that the feature information is not fused, and the fusion degree of infrared and visible light image feature information is determined by autonomous learning, thereby achieving optimal combination.

[0046] For a target detection algorithm based on deep learning, the current detection idea can be roughly divided into two categories according to the preset frame: AnchorBased and Anchor-free detection model. AnchorBased needs to preset multiple surrounding frames in advance, and realizes detection by fine-tuning the surrounding frame during prediction. AnchorBased is divided into Two-stage and One-stage detection model, and the Two-stage model detection idea is mainly divided into three parts: candidate region generation, feature extraction based on convolutional neural network, and target positioning classification. The typical network is R-CNN series network; One-stage model discards the generation process of the candidate frame, and directly fine-tunes the preset frame on the feature map to realize detection. The typical network is YOLO series, SSD and the like. The present application designs a feature pyramid detection module based on the One-stage model, takes the output of the feature fusion module as the input, adopts concat operation to splice the two features of the init module and the stage module as the detection input (i.e. fuse), and adopts multiple fusion features (fuse2~fuse5) of different dimensions to construct from top to bottom, as shown in Figure 4 For the setting of the preset frame, the k-means algorithm is used to aggregate the labeled frame in the training set to automatically generate a set of preset frames suitable for the corresponding scene.

[0047] The pyramid detection structure includes the following steps,

[0048] Step one: adjust the feature channel of the fused deep feature to be consistent with the shallow feature by point convolution;

[0049] Step two: upsample to the size of the shallow feature, and splice with the shallow feature;

[0050] Step three: the spliced feature information is subjected to convolution operation to fully fuse deep feature information;

[0051] Step four: the features of fuse5 to fuse2 are sequentially subjected to the above-mentioned upsampling splicing fusion operation, so that the detection module can fully acquire global and local feature information;

[0052] Step five: the features of four scales are respectively used to predict target categories and positions, and the prediction results are screened by a maximum value suppression algorithm to obtain optimal target detection frames, thereby realizing accurate detection of each target.

[0053] In the embodiment, experiments are respectively performed on a high-performance computer with a Titan Xp and an NVIDIA Jetson TX2 embedded platform. The network is constructed by using a caffe deep learning framework, and the feasibility and actual landing of the network are tested by using a public data set and an actual substation data set. A mini batch SGD method is used to optimize network parameters, and a Momentum momentum algorithm is used to accelerate optimization, with a momentum parameter of 0.9; an initial learning rate of 0.001, a learning strategy of step mode, that is, the learning rate is reduced by 10 times every 10,000 iterations after 30,000 iterations; a weight decay coefficient of 0.0005 is set to prevent overfitting; for the loss function, since the target detection is a multi-task model, the loss during training is the superposition of the classification loss, the confidence loss and the regression frame loss, and the specific loss function is calculated according to the YOLO network. The mean average precision (mAP) and the number of images processed per second (FPS) of the network are used to evaluate the precision and efficiency of the network. The mean average precision refers to the average value of the average precision of each category, and for different target sizes, mAP s , mAP m and mAP l are used to measure the detection effect of the network on small, medium and large targets, with 32 2 and 96 2 as the boundary, and the calculation formula is as follows:

[0054]

[0055]

[0056] Wherein, in the formula (3) (4) (5) : C is a class; N (TruePositives) C indicates the number of C class targets correctly predicted in an image; N (TotalObject) C indicates the total number of C class targets in the image; Precision C indicates the detection accuracy of C class targets in the image; N (TotalImage) C indicates the number of images containing C class targets in the training set; N (Classes) indicates the number of classes in the sample, and APC indicates the average precision of C class targets in all images. Formula (8) is an efficiency evaluation index: N is the number of images, and Tk indicates the time consumed by the network to process the kth image.

[0057] In order to effectively verify the feasibility of the method, the embodiment adopts RGBT210 dataset as a standard data for testing. The RGBT210 dataset is composed of images collected by infrared and visible light cameras with the same imaging parameters in 210 scenes. The dataset contains about 210,000 images, covering about 20 kinds of targets in infrared and visible light image pairs under different time periods and different light intensities. Since the dataset is large and most of the images are similar, in order to quickly verify the proposed network, 5000 images with low similarity are selected for testing. The selected images include 10 categories such as cars, pedestrians, bicycles, dogs, kites, etc. The normalized image size is 512x448, and the training, verification and test sets are constructed in the ratio of 7:1:2. The training and testing are carried out on a computer equipped with Titan Xp. The experiment first tests the single branch of the proposed network, that is, deletes the fusion module and the infrared branch, and only uses the visible light branch for target detection. When the network is trained, the batchsize is set to 4. After about 100,000 iterations converge, compared with the target network of the same type, the comparison results are as follows:

[0058]

[0059]

[0060] From the above table, it can be seen that the visible light single branch network constructed by the application has higher precision and efficiency than the mainstream high-precision (Faster RCNN) and high-efficiency (SSD) target networks. Shuffle+SSD) target detection network, a better balance of network performance. At the same time, the network with depth separable convolution instead of traditional convolution, and cited the residual, LeakyReLU activation function and other network construction strategy, compared with the same type of network (YOLO), the network with small precision loss to exchange network efficiency greatly improved. But relative to Faster RCNN, due to the network for single step detection, and depth separable convolution relative to traditional convolution loss part of the feature information, so as to cause the accuracy has decreased. To further verify the infrared and visible light dual branch structure of the complementary nature of the features and the effectiveness of the proposed feature fusion structure, the experiment respectively test the visible light, infrared and fusion network performance, at the same time, for the two branch fusion structure, respectively test the direct addition (Eltwise), splicing (Concat) and the present application is proposed to adapt to the fusion structure, the experimental test results as follows and Figure 5 :

[0061]

[0062] It can be seen that, due to the lack of more detailed information, compared with the visible light image, the infrared image target detection effect is poor; and visible light by light, also caused part of the target can not be identified. Through the two branch fusion method, the image feature information is more rich, network detection accuracy also has great improvement, but due to the introduction of new branch, network efficiency has certain decline. At the same time, different fusion method, network performance also has certain influence, Eltwise and Concat fusion method although increased the feature information, but also introduced more noise, and the present application is proposed to method better alleviate the introduction of noise, make two branch effective information can better complement, detection accuracy is also higher. In order to better verify the advantages of the proposed network, the experiment and other same type of infrared and visible light fusion network test comparison, detection effect as Figure 6 shown.

[0063] The test result of the substation scene test: the test result of the standard data set effectively verifies the feasibility of the proposed network, in order to further verify the generalization ability and actual landing of the network, the actual substation scene device image data is used to test and verify the network. In the actual substation scene, most of the devices need to be inspected by the infrared temperature measurement robot to monitor the device condition, but due to the complexity of the actual environment, how to avoid the interference of other devices and accurately locate the device to be measured is extremely important for the robot. Based on this, the substation inspection robot carrying Jetson TX2 is taken as the test platform, the visible light and infrared cameras carried by the robot are used to collect device images, and according to the parameters of the two cameras, the collected infrared and visible light images are cropped to align each target. The experiment mainly collects 6 kinds of substation device images such as circuit breaker, insulator, cold control box and transformer, covering different sizes of targets, 5000 images with low repetition rate are selected from them, each target in the image is manually labeled by using the public labeling tool LabelImg, and the image size is adjusted to 512x448. The processed images are constructed into training, verification and test sets in the ratio of 7:1:2, and the training test is carried out on the Jetson TX2 platform, and the detection effect is as shown in Figure 7 As can be seen from the experimental results in the actual scene, due to the lower computing performance of the test platform, the efficiency of each network is correspondingly reduced to a lower level, but overall, the network proposed in the application basically achieves the same detection efficiency compared with the same type of high efficiency network. Although the efficiency is still relatively low, but compared with the robot with slow inspection speed, it can basically meet the real-time detection requirements. At the same time, in terms of detection accuracy, due to the relatively simple data in the actual scene compared with the standard data set, the detection accuracy of each method is improved, and for simple device environment, each algorithm can basically realize accurate detection, such as Figure 7 The detection of the substation box in the first row; but for the relatively complex environment, such as Figure 7 In the second and third rows, there are situations of too large target, mutual influence of each target and complex background, which greatly affect the detection effect of each algorithm. But through overall comparison, it can be seen that the detection effect of the network proposed in the application is basically not inferior to the high-precision network, and can achieve the detection efficiency of the high-efficiency network. As can be seen from the above, the method proposed in the application has stronger robustness and generalization ability, and can be well applied to the detection of substation equipment by the inspection robot, so that the robot can complete the detection task with high precision and high efficiency.

[0064] The present application aims at the limitations of the current single visible light target detection algorithm, and proposes a deep neural network target detection method fusing infrared and visible light image feature information. The method takes deep separable convolution as the basic feature extraction unit, combines high-efficiency LeakyReLU activation function and residual structure to construct parallel infrared-visible light feature extraction branches; in order to effectively fuse the target feature information in infrared and visible light images, an adaptive feature fusion module is introduced, which uses self-learning to fuse the effective information in the two branches, avoiding the introduction of too much noise; at the same time, in order to ensure the accurate detection of the network to different size targets, the feature pyramid structure is used to fuse the feature information of different dimensions, and multiple scale features are used to predict the target, which improves the detection effect of different size targets. The experimental results in the standard data set and the actual substation equipment detection scene show that the network effectively balances the detection accuracy and efficiency, and can be better applied to the substation inspection robot to complete the equipment detection task.

[0065] Finally, it needs to be emphasized that the above only describes the preferred embodiments of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An object detection algorithm based on infrared and visible light images, characterized in that: The method comprises the following steps of: combining a deep separable convolution with a residual structure, constructing a parallel high-efficiency feature extraction network, and extracting features of infrared and visible light images. The feature fusion module models the features of the two branches by linear combination, and realizes information sharing between infrared and visible light images by using an autonomous learning method, so that the features extracted by the two branches are complementary. The detection module uses multiple deep features of different scales to construct a feature pyramid prediction structure by layer-by-layer upsampling fusion, thereby improving the detection accuracy of the network for targets of different scales. The feature extraction module is composed of an init module and multiple stage modules. The init module pre-processes the original image, extracts target significant features by using parallel convolution and pooling branches with a step size of 2, reduces the image dimension, filters part of the noise, and guarantees the in-depth extraction of features by the subsequent structure. The stage module uses a residual structure to construct a convolution layer and an activation layer, and reduces the dimension by using a 2x2 pooling operation with a step size of 2 between different stages. The stage module introduces a residual structure to avoid the problems of gradient vanishing and gradient explosion caused by too many network layers during training, and uses LeakyReLU as an activation function to reduce the probability of neuron necrosis and make the network converge faster. In the formula, α represents an offset, which is a small hyperparameter, and the default setting is 0.

02. wherein, is the pixel value of the coordinate position (i, j) on the visible light feature map to be fused, and similarly, is the pixel value of the coordinate position (i, j) on the infrared feature map to be fused, and AA AB BA BB is the weight to be learned, y is the output of the corresponding feature map after fusion, and represents updating the weight by error back propagation, and from the linear weighting calculation formula (2) learned autonomously, it can be seen that when AA BB AB BA , it indicates that the feature information is not fused, and the fusion degree of the infrared and visible light image feature information is determined by the autonomous learning manner, thereby achieving optimal combination.​​​​​​ 2. The target detection algorithm based on infrared and visible light images according to claim 1, characterized in that: The feature fusion module considers the network running efficiency, fuses the last layer of each scale in the feature extraction process, that is, fuses the pooling layer between the last layer of the init module and the stage module, and uses an autonomous learning linear weighting method to replace the direct addition of feature information, thereby avoiding the introduction of too much noise during fusion. The fusion calculation formula is as follows 3. The target detection algorithm based on infrared and visible light images according to claim 2, characterized in that: The autonomous learning linear weighting calculation formulas (1) and (2) are as follows The detection module uses the output of the feature fusion module as input, uses a concat operation to splice the features of the init module and the stage module as detection input, Meanwhile, multiple fusion features of different dimensions are used to construct a feature pyramid detection structure from top to bottom. The pyramid detection structure comprises the following steps, Step one: adjust the feature channel of the fused deep feature to be consistent with the shallow feature by point convolution; Step two: upsample to the size of the shallow feature, and splice with the shallow feature; Step three: perform convolution operation on the spliced feature information to fully fuse the deep feature information; Step four: repeatedly perform the above upsampling, splicing and fusion operations on multiple fusion features of different dimensions, so that the detection module can fully obtain global and local feature information. Step five: the target class and location are predicted by using four scales of features respectively, and the optimal target detection frame is screened out by the maximum suppression algorithm, so as to realize accurate detection of each target.

Images