A small target image detection method and system based on East algorithm

By adopting the East algorithm-based method in small object detection, using the feature fusion of FPN and bidirectional gated loop unit, the problem of difficulty in detecting small objects in the existing technology is solved, and efficient and accurate small object detection is achieved.

CN113989621BActive Publication Date: 2024-11-19GUIZHOU TUYUN SURVEYING & MAPPING PRODUCT QUALITY INSPECTION CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111428661.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-29
Publication Date
2024-11-19
Estimated Expiration
2041-11-29

AI Technical Summary

Technical Problem

Existing object detection algorithms do not perform well in small target image recognition, making it difficult to effectively detect small targets, especially in multi-scale and long-distance scenarios.

Method used

A small-target image detection method based on the East algorithm is adopted, a multi-scale feature pyramid is built using FPN, and feature fusion is combined with a bidirectional gating cyclic unit, and end-to-end training is carried out through optimization of loss function and ADAM optimizer to improve detection accuracy and speed.

Benefits of technology

It significantly improves the speed and accuracy of small object detection, which is faster than traditional algorithms and has better detection effect. It can effectively deal with multi-scale targets, reduce gradient vanishing problems and accelerate algorithm convergence.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113989621B_ABST
    Figure CN113989621B_ABST
Patent Text Reader

Abstract

The present invention discloses a small target image detection method and system based on the East algorithm. The method first establishes a target detection network, and the target detection network includes a feature extraction layer, a feature fusion layer and an output layer. A feature pyramid is constructed based on FPN in the feature extraction layer; in the feature fusion layer, the fused feature map is input into a gated cyclic unit for parameter update; finally, the network parameters are optimized by a loss function, and a target detection result is output. The present invention uses the EAST algorithm to detect small target images, and at the same time introduces an FPN network to extract more feature information, and introduces a bidirectional GRU unit to optimize parameter update and generalization performance, thereby improving the speed and accuracy of target detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a small target image detection method and system. Background Art

[0002] The detection of small targets is a common need in daily life and industrial production, such as long-distance vehicles and pedestrians in surveillance videos, distant environments that need to be perceived in autonomous driving scenarios, and various small targets from the perspective of high-altitude drones. At present, target image detection algorithms based on deep learning are mainly divided into two categories: one is based on candidate frames represented by fast RCNN, fast R-CNN, and R-FCN, and the other is based on integrated convolutional networks represented by YOLO, SSD, and retinal networks. However, these algorithms are based on general scenarios and perform well on general data sets, but the recognition effect of small target images is not ideal. Small target images are small in size, diverse in scale, and difficult to detect. The current mainstream target detection algorithms usually have a detection accuracy of more than twice that of small targets for large targets. It can be seen that the detection of small targets is extremely difficult, and the problem of small target detection has not been well solved yet. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a small target detection method and system, which can reduce the amount of calculation and improve the speed and accuracy of target detection.

[0004] Technical solution: The small target image detection method based on the East algorithm described in the present invention comprises the following steps:

[0005] (1) Inputting an image to be detected and establishing a target detection network; the target detection network includes a feature extraction layer, a feature fusion layer and an output layer, in which a feature pyramid is constructed based on FPN in the feature extraction layer; in the feature fusion layer, the fused feature map is input into a bidirectional gated recurrent unit for forward propagation, thereby reducing the gradient vanishing problem while retaining long-term sequence information;

[0006] (2) Optimize the loss function and update the network parameters;

[0007] (3) Output target detection results.

[0008] Furthermore, the loss function in step (2) includes score map loss and geometric map loss, and the score map loss adopts the focal loss function to improve the detection accuracy.

[0009] Furthermore, in step (1), shallow feature extraction and deep feature extraction are performed on the PVANET network based on FPN, and four levels of feature maps are extracted to construct a multi-scale feature pyramid. The shallow feature extraction uses 3 layers of 3*3 dimensional convolution kernels for convolution; the deep feature extraction process uses the Inceptionv1 module to decompose the convolution kernel, and uses 1*3 and 3*1 dimensional convolution kernels for convolution. Improve the small target detection performance and multi-scale target detection effect.

[0010] The small target image detection system based on the East algorithm described in the present invention comprises:

[0011] Input layer, used to input the image to be detected;

[0012] Feature extraction layer, used to construct a multi-scale feature pyramid in combination with the FPN grid;

[0013] The feature fusion layer is used to perform feature fusion and input the fused feature map into the bidirectional gated recurrent unit to reduce the gradient vanishing problem while retaining long-term sequence information;

[0014] The output layer is used to output the target detection results.

[0015] Beneficial effects: Compared with the prior art, the advantages of the present invention are: (1) the algorithm is fast, and small target image detection is performed based on the EAST (efficient and accurate scene text detection) algorithm, which is faster than the target detection algorithm based on candidate frames; (2) the target detection effect is good: FPN is introduced to fuse high-level semantic information and low-level position information, and predictions are made separately at each fusion layer to improve the small target detection performance and multi-scale target detection effect; (3) the introduction of bidirectional GRU units accelerates the convergence speed of the algorithm in time and optimizes the performance of parameter updating and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of the small target image detection method of the present invention. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0018] like Figure 1 As shown, the small target image detection method comprises the following steps:

[0019] (1) Input image

[0020] This embodiment uses the network PVANET pre-trained on the ImageNet dataset as the basic network.

[0021] (2) Shallow feature extraction

[0022] The convolutional layer of the PVANet network uses 3 layers of 3*3 dimensions and a convolution kernel with a step size of 2 for convolution; the first convolutional layer uses the ReLU activation function with an output dimension of 24; the second convolution layer uses the CReLU activation function with an output dimension of 48; the third convolutional layer uses the CReLU activation function with an output dimension of 32.

[0023] (3) Deep feature extraction

[0024] The PVANet network uses the Inceptionv1 module for deep feature extraction, and further asymmetrically decomposes the 3×3 convolution kernel into two convolution kernels of 1×3 and 3×1 dimensions, further reducing the number of network parameters and improving the nonlinear expression ability of the network by increasing the number of layers.

[0025] Four levels of feature maps are extracted to obtain multi-scale feature maps {p1, p2, p3, p4}, and the extracted feature maps are 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image size, respectively.

[0026] (4) Introduce a bidirectional GRU (Gated Recurrent Unit) and use the following feature mapping: upsample feature map p1, enlarge it to 2 times, and then cascade it with p2 along the channel axis, then use 1*1 convolution to reduce the number of channels and the amount of calculation, and then perform 3*3 convolution to fuse the information, repeat the above steps, and merge the 4 feature maps in sequence.

[0027] (5) The merged feature map is input into the bidirectional GRU. The GRU calculation formula is as follows:

[0028] z t =σ(W z [h t-1 ,X t ])

[0029] r t =σ(W r [h t-1 ,X t ])

[0030]

[0031] W z , W r , W represents the weight matrix, σ represents the sigmoid function; X t Represents the input of the gated recurrent unit at time t; represents the candidate state activation function, which is composed of the input X at time t t , reset gate t , and the activation value h at time t-1 t-1 Joint decision; trepresents the activation value of the current gated recurrent unit; tanh is the activation function; z t 、r t They represent an update gate and a reset gate respectively.

[0032] Set the output of the forward GRU at time t to be The output of the reverse GRU at time t is recorded as Connect two output states in opposite directions to the same output, outputting information about both the forward and backward states at the same time.

[0033] The bidirectional GRU unit is used to accelerate the convergence speed of the algorithm in time, optimize the performance of parameter updating and generalization, and output the results.

[0034] (6) The combined multi-channel feature maps of each feature layer are projected into a 1-channel score map and a 5-channel geometry map. The score map indicates the confidence that the pixel is an object. The geometric output is RBOX, and the geometric information includes the axis-aligned bounding box AABB of 4 channels and the rotation angle of one channel, where AABB represents the distance from the pixel position to the top, bottom, left, and right boundaries of the rectangle.

[0035] (7) Optimizing the loss function

[0036] The loss function consists of two parts, L = L s +L g , where L s represents the loss of the score map, L g Represents the loss of the geometric graph.

[0037] The focus loss function is used to reduce the impact of the difficulty of identifying positive and negative samples on the model effect. The focus loss adds an additional modulation coefficient to the standard cross entropy loss function to reduce the weight of the classified samples. The formula is as follows:

[0038] L s =-(1-p t ) r log(p t )

[0039] In the formula, p t is the probability of the predicted category, r is the parameter, r=2.

[0040] The geometric loss is the weighted sum of the AABB loss and the angle loss, and the formula is as follows

[0041] L g =L AABB +λ θ (1-cos(θ1-θ2))

[0042] Where θ1 represents the predicted rotation angle, θ2, λ θRepresent the rotation angle and weight of the actual mark respectively. In order to ensure the scale invariance of the geometric loss, the IoU (intersection over union) loss is used for the AABB part of the geometric loss.

[0043]

[0044] Where R1 represents the predicted AABB shape; R2 represents the annotation box.

[0045] The result of step (7) is fed into the network model to update the network parameters.

[0046] (8) End-to-end training using the ADAM optimizer

[0047] 426*426 images are uniformly extracted from the image set as samples, the batch size is 24, the initial ADAM learning rate is 1e-2, the decay exponent is 0.8, the decay rate is 15600, and the learning rate stops at 1e-5.

[0048] (9) Finally, the neighborhood is obtained through local-aware non-maximum suppression (LNMS), and all input sets are weighted merged in turn to obtain the merged set, suppressing the prediction box with the highest confidence score and the prediction box with a lower score to obtain the final result.

[0049] (91) for the same rectangular frame, iteratively merge the current rectangular frame with the maximum rectangular frame of the next merged rectangle;

[0050] (92) Repeat step (91) to merge the rectangular boxes row by row, and the merged set is S.

[0051] (93) Select the prediction box M with the highest confidence score from S and move it to the final detection result D.

[0052] (94) Remove the boxes between S and M in IOU that are greater than the threshold.

[0053] (95) Repeat steps (93) and (94) until the set S is empty to obtain the final detection result D.

Claims

1. A small target image detection method based on East algorithm, characterized in that: The steps include: (1) Inputting an image to be detected and establishing a target detection network; the target detection network includes a feature extraction layer, a feature fusion layer and an output layer, in which a feature pyramid is constructed based on FPN in the feature extraction layer; in the feature fusion layer, the fused feature map is input into a bidirectional gated recurrent unit for forward propagation; (2) Optimize the loss function and update the network parameters; (3) Output target detection results; In the feature extraction layer, shallow feature extraction and deep feature extraction are performed on the PVANET network based on FPN, and four levels of feature maps are extracted to construct a multi-scale feature pyramid to obtain a multi-scale feature map {p1, p2, p3, p4}; the convolution layer of the PVANet network is used to extract shallow features, and the Inceptionv1 module of the PVANet network is used to extract deep features; the convolution layer uses 3 layers of 3*3 dimensions and a convolution kernel with a step size of 2 for convolution, and the Inceptionv1 module asymmetrically decomposes the 3×3 convolution kernel into two convolution kernels of 1×3 and 3×1 dimensions; In the feature fusion layer, the bidirectional gated recurrent unit upsamples p1, amplifies it to 2 times, and then cascades it with p2 along the channel axis, and then performs 1*1 convolution and 3*3 convolution, repeats the above steps, and merges 4 feature maps in sequence; The merged feature map is input into the bidirectional GRU, and the GRU calculation formula is as follows: z t =σ(W z [h t-1 ,X t ]) r t =σ(W r [h t-1 ,X t ]) W z , W r , W represents the weight matrix, σ represents the sigmoid function; X t Represents the input of the gated recurrent unit at time t; represents the candidate state activation function, which is composed of the input X at time t t , reset gate t , and the activation value h at time t-1 t-1 Joint decision; t represents the activation value of the current gated recurrent unit; tanh is the activation function; z t 、r t They represent update gate and reset gate respectively; Set the output of the forward GRU at time t to be The output of the reverse GRU at time t is recorded as Connect two output states in opposite directions to the same output, outputting information about both the forward and backward states at the same time; In the output layer, the combined multi-channel feature maps of each feature layer are projected into a 1-channel score map and a 5-channel geometry map, the score map indicates the confidence that the pixel is a target, and the geometry map is output as RBOX.

2. The small target image detection method based on East algorithm according to claim 1, characterized in that: The loss function in step (2) includes score map loss and geometric map loss, and the score map loss adopts the focal loss function.

3. The small target image detection method based on East algorithm according to claim 1, characterized in that: In step (3), the fused feature map is input into the LNMS module for weighted merging in sequence, the target is marked with several rectangular frames, the current rectangular frame is merged with the maximum rectangular frame of the next merged rectangle one by one, the rectangular frames with intersection-over-union ratio greater than the threshold are deleted, and the rectangular frame with the highest confidence score is selected as the target detection result output.

4. A small target image detection system based on the East algorithm based on the method according to any one of claims 1 to 3, characterized in that: include: Input layer, used to input the image to be detected; Feature extraction layer, used to construct a multi-scale feature pyramid in combination with the FPN grid; The feature fusion layer is used to perform feature fusion and input the fused feature map into the bidirectional gated recurrent unit, while outputting the forward and backward state information; The output layer is used to output the target detection results.