Improved aerial target detection method and device of anchor-frame-free detector

Abstract

The invention provides an improved aviation target detection method and device for an anchor-frame-free detector. The method comprises the following steps: S1, obtaining a to-be-detected image; s2, preprocessing the to-be-detected image to obtain a preprocessed image; and S3, inputting the preprocessed image into a trained target detection model for identification to obtain a corresponding image detection result. The target detection model comprises the following modules: a feature extraction module used for performing feature extraction on the preprocessed image to obtain a feature map; the feature fusion module is used for performing feature fusion based on the feature map to obtain a fused feature map; the hierarchical fusion module is used for performing hierarchical fusion according to the fused feature map to obtain an output feature map; and the classification module is used for processing the output feature map to obtain the category confidence and bounding box coordinates of each detected target. According to the method, high-performance aerial image detection can be realized simply, quickly and easily.

Description

Technical Field

[0001] The invention belongs to the technical field of computer vision, and in particular relates to an improved aerial target detection method without an anchor frame detector. Background Art

[0002] As one of the research directions in the field of computer vision, object detection can provide valuable information for the semantic understanding of images and videos. Its essence is to locate and classify objects. Its main task is to accurately and efficiently find all objects of interest in a given image, and use rectangular bounding boxes to locate the position and size of the detected objects. It is related to object classification, semantic segmentation, and instance segmentation. Object detection plays an important role in scientific development and actual industry, such as face recognition, text detection, and logo detection.

[0003] Although existing anchor-based oriented object detection methods have achieved remarkable results, they require manual pre-set boxes, which introduces additional hyperparameters and computations. These methods usually use more complex architectures to achieve better performance, which makes them difficult to deploy on computationally constrained embedded platforms such as satellites and drones.

[0004] Traditional target detection relies on sophisticated manual feature design and extraction and has achieved certain results. However, the target detection method is generally ineffective on drone aerial images. Due to the different appearances, postures, shapes, and quantities of various target objects in the image, as well as interference from various factors such as lighting and occlusion, the target is distorted, making target detection more difficult. The main reasons are as follows:

[0005] First, the flight altitude of drones is generally high and the altitude changes quickly. The targets in the pictures taken are generally small, the dimensions change greatly, and the number of targets is large.

[0006] Secondly, due to the special shooting angle and position of drones, aerial images have the characteristics of complex background, difficult identification, and easy overlapping of targets.

[0007] Finally, drones fly at high speeds and are greatly affected by the aerial environment. Image shooting is not stable, which affects image clarity and is prone to deflection.

[0008] Based on the above points, the target detection algorithm based on deep learning has great detection difficulty and poor detection accuracy. Summary of the invention

[0009] The purpose of the present invention is to overcome the deficiencies in the prior art and to provide an improved aerial target detection method and device without an anchor box detector, which can realize high-performance aerial image detection in a simple, fast and easy-to-deploy manner.

[0010] In order to achieve the above purpose / solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0011] In a first aspect, the present invention provides an improved aerial target detection method without an anchor box detector, comprising the following steps:

[0012] Step S1, obtaining an image to be detected;

[0013] Step S2, preprocessing the image to be detected to obtain a preprocessed image;

[0014] Step S3: input the preprocessed image into a trained target detection model for recognition to obtain a corresponding image detection result.

[0015] The target detection model includes the following modules:

[0016] A feature extraction module, used to extract features from the preprocessed image to obtain a feature map;

[0017] A feature fusion module, used for performing feature fusion based on the feature map to obtain a fused feature map;

[0018] A hierarchical fusion module, used for performing hierarchical fusion according to the fusion feature map to obtain an output feature map;

[0019] The classification module is used to process the output feature map to obtain the category confidence and bounding box coordinates of each detected object.

[0020] Furthermore, the model is constructed using the FCOSR network, using ResNext50 and FPN as the backbone, and includes the following functional modules:

[0021] A feature extraction module, constructed based on the backbone, is used to extract features from the preprocessed image to obtain a feature map;

[0022] A feature fusion module, constructed based on neck, is used to perform feature fusion based on the feature map to obtain a fused feature map;

[0023] The hierarchical fusion module, which is set after the feature fusion module, improves the detection performance by utilizing features at different levels, enabling the model to better detect objects of different scales.

[0024] The classification module is used to obtain the category confidence and bounding box coordinates of each detected target, and the category confidence and bounding box coordinates of the detected target are used to identify and locate the object in the image.

[0025] Furthermore, the method of preprocessing the image to be detected to obtain a preprocessed image includes:

[0026] Adjusting the image to be detected to a uniform size;

[0027] Normalizing the pixel values ​​of the image to be detected;

[0028] Crop the image to be detected as needed;

[0029] Perform data enhancement on the image to be detected.

[0030] Furthermore, the target detection model is trained by the following steps:

[0031] Step A, obtaining a training aerial image dataset and corresponding annotation information, wherein the annotation information includes positioning supervision information and category supervision information; the positioning supervision information is a bounding box of each annotated object; the category supervision information is to assign a corresponding category label to each annotated object;

[0032] Step B, constructing an initial target detection model, and inputting the training image data set into the initial target detection model to obtain prediction box position information and category confidence;

[0033] Step C, using a quality focal loss function to calculate the positioning loss according to the prediction box position information and the positioning supervision information, and using a focal loss loss function to calculate the classification loss according to the category confidence and the category supervision information;

[0034] Step D, deriving the model parameters of the initial target detection model based on the positioning loss and the classification loss, and then updating the model parameters using back propagation to obtain updated model parameters;

[0035] Step E, determining whether the updated model parameters meet the termination condition, if it is determined to be yes, proceeding to step F, otherwise proceeding to step B;

[0036] Step F, saving the updated model parameters and loading them into the initial target detection model to form the target detection model.

[0037] Furthermore, the method for obtaining the training aerial image dataset includes:

[0038] Collect objects photographed from a high-altitude perspective to obtain an initial data set;

[0039] Preprocessing the initial data set;

[0040] Label the objects in each image in the preprocessed dataset, convert the dataset into a unified format, and create a label file containing the category and bounding box information of the target;

[0041] The converted dataset is divided into a training set, a validation set, and a test set to obtain a training aerial image dataset.

[0042] Furthermore, the method for obtaining the training aerial image dataset also includes:

[0043] After the data set is labeled, sample the data set;

[0044] The sample sampling includes an ellipse center sampling method and a fuzzy label assignment method;

[0045] The ellipse center sampling method comprises the following steps:

[0046] For each object in the image in the dataset, the center point (cx, cy) of the target is calculated based on the marked rectangular box.

[0047] The major axis and minor axis of the ellipse are set according to the size of the target to generate an ellipse area;

[0048] Random sampling is performed within the generated elliptical area to select appropriate positive and negative samples.

[0049] The fuzzy label assignment method comprises the following steps:

[0050] Get the sample features and corresponding label files in the dataset.

[0051] Use fuzzy logic to evaluate samples and calculate the fuzzy membership of each sample based on its characteristics and the current performance of the model;

[0052] Based on the fuzzy membership, samples are assigned to different training tasks.

[0053] According to the allocation strategy, the samples are screened to obtain a screened sample set, which includes high-membership samples and corresponding labels.

[0054] Furthermore, the quality focal loss function is:

[0055]

[0056]

[0057] Among them, QFL stands for quality focal loss, which is used to calculate the classification loss and measure the accuracy of the target detection algorithm for the target category;

[0058] Represents the replaced IoU, which represents the degree of overlap between the bounding box predicted by the target detection algorithm and the true bounding box.

[0059] is a parameter used to smoothly control the weighting rate;

[0060] C z * It represents the indicator function, also known as the unit function or indicative function. It takes the value of 1 when the target is met, otherwise it takes the value of 0.

[0061] Represents the number of positive samples, which refers to samples or image regions containing target objects.

[0062] Furthermore, the model parameters of the initial target detection model are derived using the positioning loss and the classification loss respectively, and then the model parameters are updated using back propagation to obtain updated model parameters, including:

[0063] The model was trained using the SGD optimizer with an initial learning rate (LR) of 0.01 and the initial learning rate was divided by 10 after the initial learning.

[0064] Set the momentum decay and weight decay to 0.9 and 0.0001 respectively.

[0065] In a second aspect, the present invention provides an improved aerial target detection device without an anchor box detector, comprising the following steps:

[0066] Input module: used to obtain the image to be detected;

[0067] Preprocessing module: used for preprocessing the image to be detected to obtain a preprocessed image;

[0068] Recognition module: used to input the pre-processed image into the trained target detection model for recognition to obtain the corresponding image detection result.

[0069] The target detection model includes the following modules:

[0070] A feature extraction module, used to extract features from the preprocessed image to obtain a feature map;

[0071] A feature fusion module, used for performing feature fusion based on the feature map to obtain a fused feature map;

[0072] A hierarchical fusion module, used for performing hierarchical fusion according to the fusion feature map to obtain an output feature map;

[0073] The classification module is used to process the output feature map to obtain the category confidence and bounding box coordinates of each detected object.

[0074] In a third aspect, the present invention provides an improved aerial target detection device without an anchor box detector, comprising a processor and a storage medium;

[0075] The storage medium is used to store instructions;

[0076] The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.

[0077] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. The target detection model of the present invention is set to fuse multi-level features, and the output feature map is processed to obtain the category confidence and bounding box coordinates of each detected target, which can realize high-performance aerial image detection in a simple, fast and easy-to-deploy manner.

[0080] 2. The target detection model uses the FCOSR network as the main architecture, which is not only simple in structure but also only contains convolutional layers, making it easier to deploy on most platforms.

[0081] 3. The task of aerial target detection in a single image faces many challenges such as arbitrary orientation, dense targets, and wide resolution range. This method can directly and accurately predict the bounding box of the target by redesigning the label assignment strategy with only a small modification to the FCOS structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 This is a multi-level sampling demonstration diagram;

[0083] Figure 2 It is a detection effect diagram.

[0084] Figure 3 It is a schematic diagram of the data processing flow of this application;

[0085] Figure 4 It is a schematic diagram of the model structure of this application. DETAILED DESCRIPTION

[0086] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0087] In the description of this embodiment, it should be noted that if the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the accompanying drawings, which is only for the convenience of describing this embodiment and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this embodiment.

[0088] Embodiment 1:

[0089] This embodiment provides an improved aerial target detection method without an anchor box detector, comprising the following steps:

[0090] Step S1, obtaining an image to be detected;

[0091] Step S2, preprocessing the image to be detected to obtain a preprocessed image;

[0092] Step S3: input the preprocessed image into a trained target detection model for recognition to obtain a corresponding image detection result.

[0093] The target detection model includes the following modules:

[0094] Feature extraction modules, such as Figure 1 As shown, it is used to extract features from the preprocessed image to obtain a feature map;

[0095] A feature fusion module, used for performing feature fusion based on the feature map to obtain a fused feature map;

[0096] A multi-level layer, used for performing hierarchical fusion according to the fused feature map to obtain an output feature map;

[0097] The classification module is used to process the output feature map to obtain the confidence of each point on the fused feature map corresponding to all categories; the final output of the classification module mainly includes the category confidence and bounding box coordinates of each detected target, and these information are combined to identify and locate objects in the image. The output result is as follows Figure 2 shown.

[0098] Specifically, Figure 4 As shown, the model is built using the FCOSR network structure, using ResNext50 and FPN as the backbone, and includes the following functional modules:

[0099] A feature extraction module, constructed based on the backbone, is used to extract features from the preprocessed image to obtain a feature map;

[0100] A feature fusion module, constructed based on neck, is used to perform feature fusion based on the feature map to obtain a fused feature map;

[0101] The hierarchical fusion module (Multi-level layer) is set after the feature fusion module. It improves the detection performance by utilizing features at different levels, enabling the model to better detect objects of different scales.

[0102] A classification module (LAM), a fuzzy label assignment strategy, is used to process the output feature map to obtain the confidence of each point on the fused feature map corresponding to all categories.

[0103] like Figure 3 As shown in the figure, the specific execution steps of the model are:

[0104] predict:

[0105] 1. Collect and preprocess the data to be predicted, ensuring that the format is consistent with that used during training.

[0106] 2. Use the trained model to extract features from the input data.

[0107] 3. Input the extracted features into the model and perform forward propagation to obtain the prediction results.

[0108] 4. Post-process the model output (such as thresholding and normalization) to generate the final prediction results.

[0109] 5. Output the prediction results for subsequent analysis or decision-making.

[0110] reasoning:

[0111] Load the trained model weights and structure.

[0112] Receive real-time or batch input data for inference.

[0113] Perform necessary preprocessing (such as scaling and standardization) on the input data.

[0114] Computing features in a model usually involves forward propagation through multiple network layers.

[0115] Generate the final inference results, including classification labels, regression values, etc.

[0116] Outputs the inference results, which may include visualization or further analysis.

[0117] The methods for obtaining the data set required for training this model include:

[0118] First, collect the data set. The data collected in this article is objects photographed from the perspective of a high-altitude aircraft.

[0119] Label the data set and label the objects in each picture

[0120] Convert the data into a unified format and create a label file containing the target category and bounding box information

[0121] Split the dataset into training, validation, and test sets, usually in a 70 / 15 / 15 or 80 / 10 / 10 ratio.

[0122] The main data processing steps of the hierarchical fusion module (Multi-level layer):

[0123] First, the fused feature map is received as input.

[0124] The feature information at different levels is fused and feature maps from multiple layers are integrated.

[0125] The feature maps of each layer are weighted to emphasize important features and suppress redundant information.

[0126] Use activation functions (such as ReLU) to process the fused feature maps and increase nonlinearity.

[0127] Generate the final output feature map for use by subsequent network layers.

[0128] The result of the classification module recognition is not a single target, but a multi-category target detection, so positioning and classification are necessary to achieve the desired effect. Output features Figure 1 It generally does not directly contain the annotated recognition results, but it will contain information related to the recognition results.

[0129] Step S2, preprocessing the image to be detected to obtain a preprocessed image, including:

[0130] Scaling: resize images to a uniform size to fit the model input requirements.

[0131] Normalization: Normalize pixel values, usually scaling them to the range of [0, 1] or [-1, 1].

[0132] Cropping: Crop the image as needed to remove unnecessary parts and enhance features in specific areas.

[0133] Data augmentation: Random rotation, flipping, adding noise, etc. to improve the generalization ability of the model.

[0134] Specifically, the training set generation method of the present application also includes:

[0135] Tag generation:

[0136] According to the truth

[0137] Tag creation:

[0138] Generate the bounding box of the target based on the annotation information, which usually includes the coordinates of the box (x, y, width, height) and the category label.

[0139] Gaussian distribution:

[0140] For some models, a Gaussian distribution can be generated based on the center of the target to help the model better learn the spatial information of the target.

[0141] The specific steps of sample collection include:

[0142] 1. Ellipse center sampling:

[0143] 1. Generate an elliptical area

[0144] According to the marked rectangular box, calculate the center point (cx, cy) of the target.

[0145] Set the major and minor axes of the ellipse according to the dimensions of the target (for example, width and height)

[0146] 2. Perform random sampling within the generated elliptical area and select appropriate positive and negative samples.

[0147] Positive samples: points inside the ellipse, usually the region of the target feature.

[0148] Negative samples: points outside the ellipse, belonging to the background area.

[0149] Function: By selecting samples that better match the target characteristics, the quality of training data can be improved, thereby improving the learning effect of the model.

[0150] The 2D Gaussian distribution is defined as follows:

[0151] Σ = R θ ⋅ Σ 0 ⋅ R θ T μ=(cx,cy) R θ =[ cos θ - sin θ sin θ cos θ ], Σ 0 = 1 12 [ w 2 0 0 h 2 ]

[0152] is the covariance matrix, is the covariance matrix when the angle is equal to 0, is the average value, is the rotation transformation matrix.

[0153] Among them, cx is the horizontal coordinate of the center point, cy is the vertical coordinate of the center point, w is the width, h is the height, and θ is the rotation angle relative to the horizontal axis;

[0154] f(x) is the probability density of a two-dimensional Gaussian distribution.

[0155]

[0156] Where X represents the 2D vector coordinates and Σ is the covariance matrix.

[0157]

[0158] (The length of the major axis of the ellipse shrinks to , the minor axis length remains unchanged. )

[0159] is the covariance matrix.

[0160] Center sampling is a strategy used to concentrate sampling points near the center of the object, which helps reduce low-quality detections and improve model performance.

[0161] However, there are two problems when directly transferring the horizontal center sampling strategy to oriented object detection.

[0162] First, the horizontal center sampling area is usually a 3×3 or 5×5 square, so the angle of the OBB (oriented bounding box) affects the sampling range.

[0163] Second, the short sides further reduce the number of sampling points for large aspect ratio objects, and they limit the range of center sampling.

[0164] To mitigate these negative effects, an ellipse center sampling method based on a two-dimensional Gaussian distribution is used. The contour of the probability density function of a two-dimensional Gaussian distribution is an elliptic curve.

[0165] The purpose of ellipse center sampling is to determine valid training samples by the center of the ellipse, rather than dividing the object into ellipses. It is mainly used to improve the training efficiency of the model and ensure that the model focuses on the valid area containing the target.

[0166] 2. Fuzzy label allocation strategy:

[0167] Different from traditional hard label assignment, such as a clear division between positive and negative samples, fuzzy label assignment allows a certain degree of fuzziness in the labels of samples.

[0168] For example, a sample can be partially regarded as a positive sample and partially as a negative sample, which can better handle the cases with blurred or overlapping boundaries.

[0169] The fuzzy sample allocation strategy is introduced, and the two-dimensional Gaussian distribution is approximately used as the distance measure from the sampling point to the target center. The center distance J(X) is defined by the formula:

[0170] By calculating J(X), we can get a score associated with the anchor box, which indicates the importance or credibility of the anchor box in a specific area X.

[0171] The anchor box segmentation method is not a direct recognition method. It is used to evaluate the effectiveness and accuracy of the anchor box.

[0172] The formula is used to train the model parameters. By defining the loss function and combining it with the probability density, the model can effectively learn the key features of the target detection task.

[0173] Specifically, a fuzzy sample allocation strategy is introduced, and a two-dimensional Gaussian distribution is approximately used as the distance measure from the sampling point to the target center. The center distance J(X) is defined by the formula:

[0174]

[0175] f(x) is the probability density of a two-dimensional Gaussian distribution.

[0176]

[0177] Where X represents the 2D vector coordinates and Σ is the covariance matrix.

[0178] For any target, calculate the J(X) value of each sampling point (J(x) is the center distance).

[0179] The larger the value of J(X), the closer X is to the target. When a sampling point is included by multiple targets at the same time, the label with the largest J(X) target is assigned to the sampling point.

[0180] The task of aerial target detection in a single image faces many challenges such as arbitrary orientation, dense targets, and wide resolution range. These problems make it difficult to effectively detect aerial targets in existing methods.

[0181] This method can directly and accurately predict the OBB of the target by redesigning the label assignment strategy with only a small modification to the FCOS structure. The assignment strategy is usually implemented in the LAM module. This module optimizes the training process of the target detection model by applying different assignment strategies.

[0182] Figure 3 In the figure, H*W*512: represents a feature map with dimensions of height (H) and width (W) and 512 channels. It is the feature map output by the convolutional layer and contains rich high-dimensional features.

[0183] H*W*1: represents a single-channel feature map with a size of H*W.

[0184] H*W*5: The dimensions are height (H) and width (W), and the number of channels is 5.

[0185] QFLOSS: QFLOSS is a loss function module for classification loss.

[0186] H*W*256, H*W*1 and H*W*5 are feature maps generated after the feature extraction network (backbone network and neck). The input is a picture of size H*W*3. After extraction in different layers, such as the backbone network, the height and width become smaller and the channels become larger. H*W*256 / H*W*512 is obtained after the multi-level layer. The feature map contains rich high-level semantic information and is suitable for classification tasks and bounding box regression. It is usually used to predict the category of the target.

[0187] H*W*1 is obtained by convolution after the feature map output by the backbone network and neck, and is used for bounding box regression or confidence prediction of target existence, indicating whether there is a target at each position. Each pixel corresponds to a value, indicating the possibility of the target.

[0188] H*W*5 is the feature map output by the backbone network and neck after 1x1 convolution

[0189] H*W*256 / H*W*512: This feature map usually comes from the last layer of classification convolution. It is responsible for outputting the probability of the target category for each pixel, usually using the softmax activation function.

[0190] H*W*1: This feature map is usually generated by a dedicated center point prediction layer to indicate whether each pixel is the center of the target, and usually outputs a single-channel binary classification probability.

[0191] H*W*5: This feature map comes from the regression convolution layer and outputs the bounding box parameters of the target. It usually contains the center coordinates (cx, cy), width (w) and height (h), which can be represented as five channels.

[0192] Specifically, the target detection model is trained through the following steps:

[0193] Step S2-1, obtaining a training aerial image dataset and corresponding annotation information, wherein the annotation information includes positioning supervision information and category supervision information; the category supervision information is to assign a corresponding category label to each annotated object;

[0194] Step S2-2, constructing an initial target detection model, and inputting the training image data set and the annotation information into the initial target detection model,

[0195] Step S2-3, using a quality focal loss function to calculate a positioning loss according to the prediction box position information and the positioning supervision information, and using a focal loss loss function to calculate a classification loss according to the confidence and the category supervision information;

[0196] Step S2-4, deriving the model parameters of the initial target detection model based on the positioning loss and the classification loss, and then updating the model parameters using back propagation to obtain updated model parameters;

[0197] Step S2-5, judging whether the updated model parameters have reached the termination condition, if it is judged to be yes, proceeding to step S2-6, otherwise proceeding to step S2-2;

[0198] Step S2-6, saving the updated model parameters and loading them into the initial target detection model to form the target detection model.

[0199] Specifically, the quality focal loss function is:

[0200]

[0201]

[0202] Among them, QFL stands for quality focal loss, which is used to calculate the classification loss. The classification loss is used to measure the accuracy of the target detection algorithm for the target category;

[0203] Represents the replaced IoU, which represents the degree of overlap between the bounding box predicted by the target detection algorithm and the true bounding box.

[0204] parameter Smoothly control the weighting rate;

[0205] C z * It represents the indicator function, also known as the unit function or indicative function. It takes the value of 1 when the target is met, otherwise it takes the value of 0.

[0206] Represents the number of positive samples, which refers to samples or image regions containing target objects.

[0207] Specifically, step S2-4, respectively using the positioning loss and the classification loss to derive the model parameters of the initial target detection model, and then using back propagation to update the model parameters to obtain updated model parameters, includes:

[0208] The model was trained using the SGD optimizer with an initial learning rate (LR) of 0.01, which was then divided by 10.

[0209] Set the momentum decay and weight decay to 0.9 and 0.0001 respectively.

[0210] This method is not only simple in structure, but also contains only convolutional layers, making it easier to deploy on most platforms.

[0211] This paper designs a high-performance aerial image detection algorithm that is simple, fast, and easy to deploy.

[0212] This embodiment proposes a one-stage anchor-free rotated object detector that can be deployed on most platforms and uses a well-defined label assignment strategy to process the characteristics of aerial image objects.

[0213] Use the ellipse center sampling method to define a suitable sampling area for the oriented bounding box.

[0214] Using a fuzzy sample assignment strategy provides reasonable labels for overlapping objects.

[0215] The network directly predicts the center point (including x and y), width, height, and rotation angle of the object.

[0216] The convergence target of the feature map is determined by LAM.

[0217] The algorithm does not introduce additional components into the architecture. The centrality branch is removed, making the network simpler and requiring less computation.

[0218] The network output contains a C-dimensional vector from the classification branch and a 5-dimensional (5D) vector from the regression branch. Give each component of the regression output a different range. The offset can be negative, but the width and height must be positive, and the angle must be constrained to be between 0 and 90 degrees.

[0219] The loss functions include classification loss and regression loss.

[0220] Quality focus loss ( ) is used for classification loss, mainly to remove the centrality branch from the original algorithm.

[0221] The regression loss uses IoU loss.

[0222] It is a part of the General Focal Loss (GFL). It unifies the training and testing process by replacing the one-hot label with the IoU value between the prediction and the ground truth. QFL suppresses low-quality detection results and also improves the performance of the model.

[0223] Given below is defined as:

[0224]

[0225] in represents the replaced IoU, and the parameter Smoothly control the weighting rate. It mainly represents the IoU between OBBs through the distance between two-dimensional Gaussian distributions.

[0226]

[0227] in Indicates the number of positive samples.

[0228] All positions on the multi-level feature map Calculate the sum above.

[0229] The indicator function is ,if 1 if the value is true, 0 otherwise.

[0230] ResNext50 and FPN are used as the backbone of FCOSR.

[0231] The model is trained using the SGD optimizer with an initial learning rate (LR) of 0.01 and the LR is divided by 10 afterwards.

[0232] The momentum decay and weight decay are 0.9 and 0.0001 respectively.

[0233] Embodiment 2:

[0234] This embodiment provides an improved aerial target detection device without an anchor frame detector, comprising the following steps:

[0235] Input module: used to obtain the image to be detected;

[0236] Preprocessing module: used for preprocessing the image to be detected to obtain a preprocessed image;

[0237] Recognition module: used to input the pre-processed image into the trained target detection model for recognition to obtain the corresponding image detection result.

[0238] The target detection model includes the following modules:

[0239] A feature extraction module, used to extract features from the preprocessed image to obtain a feature map;

[0240] A feature fusion module, used for performing feature fusion based on the feature map to obtain a fused feature map;

[0241] A hierarchical fusion module, used for performing hierarchical fusion according to the fusion feature map to obtain an output feature map;

[0242] The classification module is used to process the output feature map to obtain the category confidence and bounding box coordinates of each detected object.

[0243] The device of this embodiment can be used to implement the method described in Embodiment 1.

[0244] Embodiment 3:

[0245] This embodiment provides an improved aerial target detection device without an anchor frame detector, including a processor and a storage medium;

[0246] The storage medium is used to store instructions;

[0247] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.

[0248] Embodiment 4:

[0249] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in Embodiment 1 are implemented.

[0250] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0251] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0252] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0253] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0254] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An improved aerial target detection method without anchor box detector, characterized in that: The following steps are involved: Step S1, obtaining an image to be detected; Step S2, preprocessing the image to be detected to obtain a preprocessed image; Step S3, inputting the preprocessed image into a trained target detection model for recognition to obtain a corresponding image detection result; The target detection model includes the following modules: A feature extraction module, used to extract features from the preprocessed image to obtain a feature map; A feature fusion module, used for performing feature fusion based on the feature map to obtain a fused feature map; A hierarchical fusion module, used for performing hierarchical fusion according to the fusion feature map to obtain an output feature map; The classification module is used to process the output feature map to obtain the category confidence and bounding box coordinates of each detected object.

2. The improved aerial target detection method without anchor box detector according to claim 1, characterized in that: The target detection model is built using the FCOSR network, using ResNext50 and FPN as the backbone, and includes the following functional modules: A feature extraction module, constructed based on the backbone, is used to extract features from the preprocessed image to obtain a feature map; A feature fusion module, constructed based on neck, is used to perform feature fusion based on the feature map to obtain a fused feature map; The hierarchical fusion module is set after the feature fusion module and is used to fuse features of different levels so that the model can better detect objects of different scales; The classification module includes a regression branch and a classification branch, which are used to obtain the category confidence and bounding box coordinates of each detected target, and the category confidence and bounding box coordinates of the detected target are used to identify and locate the object in the image.

3. The improved aerial target detection method without anchor box detector according to claim 1, characterized in that: The method of obtaining a preprocessed image by preprocessing the image to be detected comprises: Adjusting the image to be detected to a uniform size; Normalizing the pixel values ​​of the image to be detected; Crop the image to be detected as needed; Perform data enhancement on the image to be detected.

4. The improved aerial target detection method without anchor box detector according to claim 1, characterized in that: The target detection model is trained by the following steps: Step A, obtaining a training aerial image dataset and corresponding annotation information, wherein the annotation information includes positioning supervision information and category supervision information; the positioning supervision information is a bounding box of each annotated object; the category supervision information is to assign a corresponding category label to each annotated object; Step B, constructing an initial target detection model, and inputting the training image data set into the initial target detection model to obtain prediction box position information and category confidence; Step C, using a quality focal loss function to calculate the positioning loss according to the prediction box position information and the positioning supervision information, and using a focal loss loss function to calculate the classification loss according to the category confidence and the category supervision information; Step D, deriving the model parameters of the initial target detection model based on the positioning loss and the classification loss, and then updating the model parameters using back propagation to obtain updated model parameters; Step E, determining whether the updated model parameters meet the termination condition, if it is determined to be yes, proceeding to step F, otherwise proceeding to step B; Step F, saving the updated model parameters and loading them into the initial target detection model to form the target detection model.

5. The improved aerial target detection method without anchor box detector according to claim 4, characterized in that: The method for obtaining the training aerial image dataset includes: Collect objects photographed from a high-altitude perspective to obtain an initial data set; Preprocessing the initial data set; Label the objects in each image in the preprocessed data set, convert the data set into a unified format, and create a label file, wherein the label file contains the label information; The converted dataset is divided into a training set, a validation set, and a test set to obtain a training aerial image dataset.

6. The improved aerial target detection method without anchor box detector according to claim 5, characterized in that: The method for obtaining the training aerial image data set also includes: After the data set is labeled, sample the data set; The sample sampling includes an ellipse center sampling method and a fuzzy label assignment method; The ellipse center sampling method comprises the following steps: For each object in the image in the dataset, calculate the center point of the target based on the marked rectangular box; The major axis and minor axis of the ellipse are set according to the size of the target to generate an ellipse area; Perform random sampling within the generated elliptical area and select positive and negative samples; The fuzzy label assignment method comprises the following steps: Get the sample features and corresponding label files in the dataset; Use fuzzy logic to evaluate samples and calculate the fuzzy membership of each sample based on its characteristics and the current performance of the model; Based on fuzzy membership, samples are assigned to different training tasks; According to the allocation strategy, the samples are screened to obtain a screened sample set, which includes high-membership samples and corresponding label files.

7. The improved aerial target detection method without anchor box detector according to claim 4, characterized in that: The quality focal loss function is: ； ； Among them, QFL stands for quality focal loss, which is used to calculate the classification loss and measure the accuracy of the target detection algorithm for the target category; represents the replaced IoU, which represents the degree of overlap between the predicted bounding box and the true bounding box. is a parameter used to smoothly control the weighting rate; C z * It represents the indicator function, also known as the unit function or indicative function, which takes the value of 1 when the target is met, otherwise it takes the value of 0; Represents the number of positive samples, which are samples or image regions containing target objects.

8. The improved aerial target detection method without anchor box detector according to claim 4, characterized in that: The method of respectively using the positioning loss and the classification loss to derive the model parameters of the initial target detection model, and then using back propagation to update the model parameters to obtain updated model parameters includes: The model was trained using the SGD optimizer, with the initial learning rate set to 0.01, and the initial learning rate was divided by 10 after the initial learning. Set the momentum decay and weight decay to 0.9 and 0.0001 respectively.

9. An improved aerial target detection device without anchor box detector, characterized in that: The following steps are involved: Input module: used to obtain the image to be detected; Preprocessing module: used for preprocessing the image to be detected to obtain a preprocessed image; Recognition module: used to input the pre-processed image into the trained target detection model for recognition, and obtain the corresponding image detection result; The target detection model includes the following modules: A feature extraction module, used to extract features from the preprocessed image to obtain a feature map; A feature fusion module, used for performing feature fusion based on the feature map to obtain a fused feature map; A hierarchical fusion module, used for performing hierarchical fusion according to the fusion feature map to obtain an output feature map; The classification module is used to process the output feature map to obtain the category confidence and bounding box coordinates of each detected object.

10. An improved aerial target detection device without anchor box detector, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-8.

Images