Rotating target detection method and system based on phase shift coding and directional feature enhancement

Abstract

The invention discloses a rotating target detection method and system based on phase shift coding and directional feature enhancement, and belongs to the technical field of image recognition. The method comprises the steps of obtaining an image data set and performing enhancement processing; inputting into an ARC network to extract features of a target in any direction; performing adaptive feature optimization on the features through an EMA module, encoding optimized feature direction information by using ARF, generating direction sensitive features for bounding box regression, and extracting directional features for classification through a rotation invariant pooling layer; linearly mapping the angle of the rotating frame into a phase of a corresponding frequency by using a phase shift encoder, predicting a phase shift cosine value, and decoding to obtain a final phase; and finally, removing overlapped detection frames by applying a non-maximum suppression method, and reserving an optimal detection result. Wherein a Pareto optimization theory is used for adaptively weighting a loss item, model parameters are updated through a multi-gradient descent algorithm, and the obtained final model can improve the accuracy and efficiency of detection of a target containing a multi-directional small object.

Description

Technical Field

[0001] The invention belongs to the technical field of image recognition, and in particular relates to a rotating target detection method and system based on phase shift coding and directional feature enhancement. Background Art

[0002] Object detection is an important research field in computer vision. With the rapid development of computer hardware and deep learning methods, more and more object detection algorithms based on deep learning methods have emerged, such as two-stage algorithms represented by the RCNN series of algorithms and single-stage algorithms represented by the RetinaNet and YOLO series of algorithms. The above algorithms have achieved great success in most natural scenes and are widely used in fields such as autonomous driving, medical image recognition, and security monitoring. However, the above algorithms all use horizontal bounding boxes (HBB) to locate targets, which often have low accuracy and poor generalization performance when detecting targets with complex backgrounds, different orientations, dense distribution, and large aspect ratios.

[0003] In recent years, the application of object detection based on arbitrary orientation in remote sensing image detection, scene text recognition, face detection and retail scene detection has attracted more and more attention from the computer vision community. At present, most rotation object detection algorithms achieve the detection of rotating objects by introducing angle regression parameters to represent the rotating rectangle.

[0004] However, when the rotation box is at the boundary of the angle definition range, the loss of the regression-based angle prediction method increases sharply and cannot be continuous, making the regression form of the model inconsistent in boundary and non-boundary cases, thereby increasing the difficulty of model training and prediction under boundary conditions. For rotation bounding box detection, the accuracy of angle prediction is crucial. A slight angle deviation will cause a serious drop in IoU, resulting in inaccurate object detection, especially in the case of large aspect ratios.

[0005] Secondly, since the convolution operation has limited generalization ability for target rotation and scale changes, and in the application scenario of rotation target detection, the target scale changes greatly and in different directions, it is necessary to ensure certain direction and scale invariance when extracting features. When facing densely arranged targets, especially densely arranged targets with large aspect ratios, it is important to extract regional features with rotation invariance and eliminate the misalignment between regional features and targets. Summary of the invention

[0006] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a rotating target detection method and system based on phase shift encoding and directional feature enhancement, which solves the problems in the prior art.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] The rotating target detection method based on phase shift encoding and directional feature enhancement includes the following steps:

[0009] Get image dataset;

[0010] Perform data augmentation on image datasets;

[0011] The data-enhanced image is input into the target detection model for training; the target detection model includes: ARC network, EMA module, ARF, rotation-invariant pooling layer and phase shift encoder; the data-enhanced image is input into the ARC network to extract the features of targets in any direction, the EMA module performs adaptive feature optimization on the features, the ARF encodes the optimized feature direction information, generates direction-sensitive features for bounding box regression, and the rotation-invariant pooling layer extracts directional features from the direction-sensitive features for classification; the phase shift encoder linearly maps the angle of the rotating frame to the phase of the corresponding frequency, predicts the phase shift cosine value, and decodes to obtain the final phase;

[0012] Use Pareto optimization theory to adaptively weight each loss term, construct the total loss function of the target detection model, and update the target detection model parameters through the multi-gradient descent algorithm;

[0013] The trained target detection model is used to predict the position, size and rotation angle of each target, and the final phase is obtained by decoding the cosine values ​​of multiple different phase shifts.

[0014] Based on the final phase, the non-maximum suppression method is applied to remove overlapping detection boxes and retain the best detection results.

[0015] Further, the ARC network includes an ARConv module and an ARC-C2f module;

[0016] The ARConv module replaces the standard 2D convolution in the convolution module Conv with an adaptive rotation convolution, retaining its batch normalization layer and SiLU activation function layer;

[0017] The ARC-C2f module replaces the 3×3 convolution module in the bottleneck network of the C2f module with an ARConv module with the same size convolution kernel, while retaining its input and output 1×1 convolution modules.

[0018] Furthermore, the EMA module uses a multi-scale attention structure composed of parallel branches of two convolutions with different receptive fields, and captures the pixel-level attention of the input feature map without reducing the channel dimension by feature grouping and cross-scale feature fusion; for any given input feature The multi-scale attention module divides X into G groups of sub-features according to the channel dimension. After the division, the input feature is expressed as:

[0019]

[0020] In the formula, GAP w and GAP h Represent the global average pooling on the horizontal and vertical axes respectively, F1 represents the 1×1 convolution function, F3 represents the 3×3 convolution function, and σ(·) is the sigmiod function;

[0021] Output of 1×1 branch and the output of the 3×3 branch The manifestation is:

[0022]

[0023] Among them, GAP represents the global average pooling on the entire plane;

[0024] The final output Y of the EMA module is i for:

[0025]

[0026] Further, the phase shift encoder includes a mapping process, an encoding process and a decoding process;

[0027] The mapping process is:

[0028]

[0029] Where θ is the rotation angle of the bounding box, is the corresponding phase;

[0030] The encoding process is to encode the phase into cosine values ​​of different phase shifts. The encoding formula is:

[0031]

[0032] Among them, x n It represents a series of cosine values ​​calculated during the encoding process, N step is the number of phase shift steps;

[0033] The decoding process is the process of inversely deducing the phase from the cosine value of the multi-step phase shift. The calculation formula is:

[0034]

[0035] Furthermore, the total loss function L of the target detection model is:

[0036] L=ω1L cls+ω2L box +ω3L ang

[0037]

[0038] L ang =|f p -f t |

[0039] Among them, ω1, ω2, ω3 are hyperparameters for adjusting the loss weights of each task; L cls is the classification loss, is the binary cross entropy loss; L box is the bounding box coordinate regression loss L box , L ang is the angle loss for phase shift encoding; IOU is the intersection-over-union ratio between the predicted box and the real box, (x1, y1) and (x2, y2) are the coordinates of the center points of the predicted box and the real box respectively, d is the distance between the center points of the predicted box and the real box, w1, h1 and w2, h2 are the width and height of the predicted box and the real box respectively, and α is a hyperparameter for balancing the difference in aspect ratio; f t is the rotation angle θ t Encode to get; f p is the cosine of the predicted phase shift.

[0040] Furthermore, the process of applying the non-maximum suppression method to remove overlapping detection frames includes:

[0041] Sort the detection frames according to their confidence scores, select the detection frame with the highest confidence as the reference frame, and calculate its intersection-over-union ratio with other detection frames;

[0042] If the intersection-over-union ratio of other detection frames with the reference frame exceeds a preset threshold, they are considered to have detected the same target and these detection frames are suppressed. After iterations, until all detection frames are processed, the detection frame with the highest confidence and not suppressed is output as the final result.

[0043] The rotating target detection system based on phase shift encoding and directional feature enhancement includes:

[0044] Image data acquisition unit: acquires image data set;

[0045] Data enhancement unit: perform data enhancement on image datasets;

[0046] Model training unit: The data-enhanced image is input into the target detection model for training; the target detection model includes: ARC network, EMA module, ARF, rotation-invariant pooling layer and phase shift encoder; the data-enhanced image is input into the ARC network to extract the features of targets in any direction, the EMA module performs adaptive feature optimization on the features, the ARF encodes the optimized feature direction information, generates direction-sensitive features for bounding box regression, and the rotation-invariant pooling layer extracts directional features from the direction-sensitive features for classification; the phase shift encoder linearly maps the angle of the rotating frame to the phase of the corresponding frequency, predicts the phase shift cosine value, and decodes to obtain the final phase;

[0047] Model optimization unit: Use Pareto optimization theory to adaptively weight each loss term, construct the total loss function of the target detection model, and update the target detection model parameters through a multi-gradient descent algorithm;

[0048] Phase prediction unit: Use the trained target detection model to predict the position, size and rotation angle of each target, and decode the cosine values ​​of multiple different phase shifts to obtain the final phase;

[0049] Detection result screening unit: Based on the final phase, the non-maximum suppression method is applied to remove overlapping detection frames and retain the best detection results.

[0050] A computer storage medium stores a readable program, which can execute the above-mentioned rotating target detection method based on phase shift encoding and directional feature enhancement when the program is running.

[0051] An electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0052] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned rotating target detection method based on phase shift encoding and directional feature enhancement.

[0053] A computer program product includes computer instructions, wherein the computer instructions instruct a computing device to execute operations corresponding to the above-mentioned rotating target detection method based on phase shift encoding and directional feature enhancement.

[0054] Beneficial effects of the present invention:

[0055] 1. The present invention realizes feature extraction and optimization of targets in any direction through an adaptive rotation convolution network (ARC network) and a directional feature enhancement module, thereby improving the accuracy and efficiency of the model in detecting small objects in multiple directions.

[0056] 2. The present invention uses a multi-scale attention module (EMA module) to perform adaptive feature optimization on the input features, thereby enhancing the model's ability to capture key features, while suppressing irrelevant features and improving detection accuracy.

[0057] 3. The present invention encodes the optimized feature direction information through an active rotation filter (ARF) to generate direction-sensitive features for bounding box regression, thereby enhancing the model's ability to recognize the target direction and improving the accuracy of rotated target detection.

[0058] 4. The present invention uses a rotation-invariant pooling layer to extract direction-invariant features from direction-sensitive features for classification, which ensures the robustness of the classification task and reduces the impact caused by target rotation.

[0059] 5. The present invention introduces a phase shift encoder to linearly map the angle of the rotating frame to the phase of the corresponding frequency, and obtains the final phase by predicting the cosine values ​​of multiple different phase shifts and decoding them, thereby solving the boundary discontinuity problem of the rotating frame and improving the continuity and accuracy of angle prediction.

[0060] 6. The present invention applies the Pareto optimization theory to adaptively weight the loss term and updates the model parameters through a multi-gradient descent algorithm, so that the model can balance the weights of different loss functions under the multi-task learning framework, thereby optimizing the overall performance of the model.

[0061] 7. Based on the final phase, the present invention applies a non-maximum suppression method to remove overlapping detection frames, thereby retaining the best detection results, reducing redundant detection, and improving detection efficiency.

[0062] 8. The method of the present invention is implemented through electronic devices, so that rotating target detection can be deployed on various hardware platforms, which enhances the practicality and flexibility of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 It is an overall flow chart of the target detection method of the present invention;

[0065] Figure 2 It is a network structure diagram of the target detection method of the present invention;

[0066] Figure 3 is a structural diagram of ARC-C2f of the present invention;

[0067] Figure 4 It is a rotation flow chart of the convolution kernel of the present invention;

[0068] Figure 5 is a flow chart of the inference of the phase shift encoder of the present invention;

[0069] Figure 6 It is a regression branch improvement diagram of the phase shift coding of the present invention;

[0070] Figure 7 is a visualization result diagram of the model features of the present invention;

[0071] Figure 8 It is a comparison chart of the detection results of the present invention and the baseline model. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] Example 1

[0074] like Figure 1 As shown, the rotating target detection method based on phase shift encoding and directional feature enhancement includes the following steps:

[0075] S1, obtain image dataset;

[0076] For the DOTA dataset, the present invention divides the training, validation and test images into 1024×1024 sub-images with an overlap of 200 pixels. The training set of the segmented DOTA dataset contains 15,749 images, the validation set contains 5,297 images, and the test set contains 10,833 images.

[0077] For the HRSC2016 dataset, each image is scaled to 800×800 for training and testing.

[0078] S2, data augmentation for image dataset;

[0079] Data augmentation strategies include: HSV augmentation, random flip augmentation (RandomFlip) and mosaic augmentation (Mosaic). Mosaic augmentation will be turned off in the last 10 rounds of training. To ensure fairness, all experiments did not use random rotation augmentation (RR), multi-scale training and testing (MS), and test-time data augmentation (TTA).

[0080] S3, the data-enhanced image is input into the target detection model for training; the target detection model includes: adaptive rotation convolution network (ARC network), directional feature enhancement module, phase shift encoder; the directional feature enhancement module includes: multi-scale attention module (EMA module), active rotation filter (ARF), rotation invariant pooling layer;

[0081] The data-enhanced image is input into an adaptive rotation convolutional network to extract features of targets in any direction. The feature is then adaptively optimized through an efficient multi-scale attention module, and the optimized feature direction information is encoded using an active rotation filter to generate direction-sensitive features for bounding box regression. The directional features are then extracted from the direction-sensitive features through a rotation-invariant pooling layer for classification. A phase shift encoder is introduced to linearly map the angle of the rotating box to the phase of the corresponding frequency, predict the phase shift cosine value, and decode to obtain the final phase.

[0082] 1) Adaptive Rotational Convolutional Network

[0083] Due to the limited generalization ability of convolution operations for target rotation and scale changes, traditional backbone feature extraction networks cannot extract high-quality features for oriented targets, and there is a serious problem of feature misalignment. Some recent works have alleviated the problem of feature misalignment by designing feature alignment modules FAM, feature selection modules FSM or feature refinement modules FRM. Although these works have played an effective role, they only perform secondary correction on the features extracted by the backbone network and do not really solve the problem of inaccurate feature extraction by the backbone network. Therefore, in order to improve the backbone network's feature extraction ability for targets in any direction, it is necessary to use a more efficient feature extraction network (adaptive rotation convolution network).

[0084] Adaptive rotation convolutional network Figure 2 As shown in the blue dotted box below, the native YOLOv8 backbone network is reconstructed using adaptive rotation convolution, including: ARConv module and ARC-C2f module;

[0085] The ARConv module replaces the standard 2D convolution in the convolution module Conv with an adaptive rotation convolution (ARC), retaining its batch normalization layer and SiLU activation function layer.

[0086] The structural diagram of the ARC-C2f module is as follows Figure 3 As shown in Figure 1, the ARC-C2f module replaces the 3×3 convolution module in the bottleneck network of the C2f module with an ARConv module with a convolution kernel of the same size, and retains its input and output 1×1 convolution modules.

[0087] In adaptive rotation convolution (ARC), the convolution kernel is adaptively rotated to extract target features in different directions in different images, and a routing function is introduced to adapt to the large directional changes of objects in the image, so that the network can flexibly detect objects in different directions. The adaptive rotation convolution has n weights [W1,…,W n ] convolution kernel, the shape of each convolution kernel is: [C out ,C in ,k,k]. For a given input feature x, the routing function f outputs a set of rotation angles [θ1,…,θ n ] and its corresponding combination weights [λ1,…,λ n ] as follows:

[0088] θ,λ=f(x)#(1)

[0089] The n convolution kernels in the ARConv module are first rotated according to the predicted rotation angle θ = [θ1, θ2, …, θ n ]Single rotation:

[0090] W′ i =Rotate(W i θ i ),i=1,2,…,n#(2)

[0091] Among them, θ i is the convolution kernel W i The rotation angle, W' i is the rotated convolution kernel, Rotate(·) is Figure 4 The rotation process. Figure 4 The rotation process of the 3×3 convolution kernel is shown. Figure 4 (a) represents the initial convolution kernel, (b) represents the mapping of the kernel in the convolution space, (c) represents adjusting the coordinate axis to adapt to the rotation, (d) represents sampling parameters from the new space, and (e) represents the convolution kernel after rotation adjustment.

[0092] A simple usage of these rotation kernels is to convolve them with the input feature maps individually and add the output feature maps together element-wise. For a given input feature x, the output feature y of the ARConv module is calculated as follows:

[0093]

[0094] x″=BN(x')

[0095]

[0096] The number of rotated convolution kernels in ARC is controlled by the hyperparameter n. In the ARConv module, the number of convolution kernels of all ARCs is 4; while in the ARC-C2f module, the number of convolution kernels of all ARCs is set to 1.

[0097] 2) Directional feature enhancement module

[0098] After the backbone network extracts high-quality features of targets in any direction and performs feature fusion through the bottleneck layer, the present invention proposes a directional feature enhancement module to further optimize and divide the features, and provide task-differentiated features for bounding box regression and classification tasks, thereby improving the detection performance of the model. The directional feature enhancement module includes: a multi-scale attention module (EMA module), an active rotation filter (ARF), and a rotation-invariant pooling layer;

[0099] The EMA module is introduced to adaptively optimize the input features. The EMA module uses a multi-scale attention structure composed of parallel branches of two convolutions with different receptive fields, and captures the pixel-level attention of the input feature map without reducing the channel dimension by feature grouping and cross-scale feature fusion, thereby enhancing important features and suppressing irrelevant features. For any given input feature EMA divides X into G groups of sub-features according to the channel dimension. After the division, the input features can be expressed as follows:

[0100]

[0101] In the formula, GAP w and GAP h Represent the global average pooling on the horizontal and vertical axes respectively, F1 represents the 1×1 convolution function, F3 represents the 3×3 convolution function, and σ(·) is the sigmiod function;

[0102] Therefore, the output of the 1×1 branch and the output of the 3×3 branch It can be expressed as follows:

[0103]

[0104] Among them, GAP represents the global average pooling on the entire plane;

[0105] The final output Y of the EMA module is i It can be expressed as:

[0106]

[0107] Considering the cross-spatial information aggregation method, precise location information is embedded into EMA while modeling long-range dependencies. Fusing contextual information at different scales enables CNN to produce better pixel-level attention on high-level feature maps. Subsequently, by using the cross-spatial learning method, the parallelization of convolution kernels appears to be a more powerful structure to handle short-term and long-term dependencies. In contrast to the progressive behavior of limited receptive field formation, the parallel use of 3×3 and 1×1 convolutions in the intermediate feature maps can obtain more contextual information.

[0108] The active rotation filter (ARF) is used to explicitly encode the directional information of the input features. ARF is a k×k×N filter that is actively rotated N-1 times during the convolution process to produce a feature map with N directional channels, and its position and direction are explicitly encoded into a single feature map with multiple directional channels.

[0109] The direction-sensitive features activated by ARF are used for bounding box regression, and the direction-sensitive features are aggregated through the rotation-invariant pooling layer to obtain rotation-invariant features for classification. This can be achieved by selecting the direction channel with the strongest response in the direction-sensitive features as the output feature. For a given input feature x, the output of the directional feature enhancement module is calculated as follows:

[0110] x′=EMA(x),

[0111]

[0112] Wherein, EMA(·) represents the calculation process of the EMA module, which is described in detail in formulas (4) to (6). The active rotation filter F rotates clockwise along θ i The resulting variant, and x' (n) The filters are and feature map x' (n) The nth direction channel of Y, max(·) is the operation of taking the maximum value on the direction channel, cls is the input feature of the classification branch of the detection head, Y reg It is the input feature of the regression branch of the detection head (including the bounding box branch and the phase shift branch).

[0113] 3) Phase shift encoder

[0114] A phase shift encoder is used to linearly map the true angle of the rotating frame to the phase of the corresponding frequency. The final phase is obtained by predicting the cosine values ​​of multiple different phase shifts and decoding them. This continuously differentiable angle encoding method solves the boundary discontinuity problem of the rotating frame.

[0115] like Figure 5As shown, the core process of the phase shift encoder includes three parts: mapping process, encoding process and decoding process.

[0116] 3.1) Mapping process

[0117] A single sine or cosine value cannot be decoded because the period of sine or cosine is 2π, and multiple angles may correspond to one period. The angle definition period of a rectangular box is usually π (long side representation), so a mapping is needed to match.

[0118] φ=2θ#(9)

[0119] Among them, θ is the rotation angle of the bounding box, and φ is its corresponding phase. Taking the long side representation as an example, the definition range of θ is The definition range of φ is [-π,π).

[0120] 3.2) Encoding process

[0121] The encoding process is to encode the phase into cosine values ​​of different phase shifts; the encoding formula is as follows:

[0122]

[0123] Among them, x n Represents a series of cosine values ​​calculated during the encoding process. N step is the number of phase shift steps, i.e., the hyperparameter that controls the encoding length. In theory, two-step phase shift can determine a phase, but in practice, when two-step phase shift is used, there is a problem of unstable training. Therefore, a phase shift of three or more steps is generally selected, i.e., N step ≥3.

[0124] 3.3) Decoding process

[0125] The decoding process is the process of inversely deducing the phase from the cosine value of the multi-step phase shift. The calculation formula is as follows:

[0126]

[0127] The phase-shift coded detection head regression branch of YOLO-PO is improved on the regression branch of the baseline model YOLOv8. The original YOLOv8 regression branch predicts the discretization of the offset from each position to the four sides of the bounding box in the regression feature map, and then uses the formula After calculating the expectation, we get the value of (l, t, r, b), where l, t, r, b are the offsets from the position to the left, top, right, and bottom sides of the bounding box, respectively. Then we convert it into the form of center point and width and height, i.e. (x, y, w, h). For the feature map with coordinates (x a ,y a) position a, its associated prediction box is transformed as follows:

[0128] x p =x a +(rl) / 2

[0129] y p =y a +(bt) / 2

[0130] w p =l+r

[0131] h p =t+b#(12)

[0132] Figure 6 (a) and (b) are YOLO-PO and YOLOv8 regression branches, respectively. As can be seen from the figure, YOLO-PO adds a phase shift regression sub-branch based on the YOLOv8 regression branch to predict multi-step phase shift cosine values. After decoding it, we get the angle value θ of the rotating frame p . Then (l, t, r, b, θ p ) is transformed into the rotation box representation of (x, y, w, h, θ) of the five-parameter method. a ,y a ), the associated rotation prediction box is transformed as follows:

[0133]

[0134] w p =l+r

[0135] h p =t+b#(13)

[0136] Among them, N step is the number of steps of the encoding phase shift, that is, the hyperparameter controlling the encoding length, which is set to 3 here. Although θ is used when transforming the output form of the bounding box p , but during the training process, θ p When calculating the loss, the gradient is not calculated, but it is regarded as a constant. The angle loss is only related to the cosine value of the phase shift before decoding.

[0137] S4, using Pareto optimization theory to adaptively weight each loss term, constructing the total loss function of the target detection model, and updating the target detection model parameters through the multi-gradient descent algorithm;

[0138] The process of constructing the total loss function is:

[0139] S41, the active rotation filter generates a direction-sensitive feature containing target position and direction information for bounding box regression, and calculates the bounding box coordinate regression loss L box , extract direction-invariant features through the rotation-invariant pooling layer for classification, and calculate the classification loss L cls ;

[0140] Classification loss L cls Consistent with the classification loss of YOLOv8, both use BCE Loss, namely Binary Cross-Entropy Loss.

[0141] The regression loss is also consistent with YOLOv8, using CIOU Loss and bounding box coordinate regression loss L box The calculation formula is as follows:

[0142]

[0143] Among them, IOU is the intersection-over-union ratio between the predicted box and the true box, (x1, y1) and (x2, y2) are the coordinates of the center points of the predicted box and the true box respectively, d is the distance between the center points of the predicted box and the true box, w1, h1 and w2, h2 are the width and height of the predicted box and the true box respectively, and α is a hyperparameter used to balance the difference in aspect ratio.

[0144] S42, introduce implicit coding, use the phase shift regression sub-branch to output the predicted phase shift cosine value, and use the angle coding value to calculate the angle loss L of the phase shift coding ang ;

[0145] Using the encoded value of the angle for loss calculation is called implicit encoding. Directly embedding the decoding function into the feedforward network of the detection head and using the angle value for regression loss calculation is called explicit encoding. Implicit encoding loss calculation is currently the mainstream approach for angle-encoded rotation target methods.

[0146] As shown in formula (10), the result of the encoding function output is the cosine value of the phase shift, and its value range is in the range of [-1,1]. In order to make the training more stable, the output of the phase shift regression sub-branch is transformed:

[0147] f p =2×sigmoid(X feat )-1#(14)

[0148] Among them, X feat The feature map of the last convolution output of the phase shift regression sub-branch is scaled to the range of [-1, 1] using the formula, f p is the cosine of the predicted phase shift.

[0149] For the ground truth (GT) box, its rotation angle is θ t , for θ t Encode to get f t Then, use L1 Loss to calculate the loss for both, the formula is as follows:

[0150] L ang =|f p -f t |#(15)

[0151] When the present invention initially tried to use explicit coding to calculate the loss, the loss was very unstable during the training process, continued to oscillate without convergence, and the accuracy dropped seriously. Therefore, the decoded angle value θ was used in the training process. p It is considered as a constant and is only used to transform the output without calculating its gradient.

[0152] S43, such as Figure 2 As shown on the right side of the figure, the Pareto optimization theory is used to adaptively weight each loss term to construct the total loss function of the target detection model:

[0153] L=ω1L cls +ω2L box +ω3L ang #(16)

[0154] Among them, (ω1,ω2,ω3) are the hyperparameters for adjusting the loss weights of each task; L cls is the classification loss, L box is the bounding box coordinate regression loss L box , L ang The angle loss is the phase shift encoding; the classification loss is consistent with the classification loss of YOLOv8, both using BCE Loss. The regression loss is also consistent with YOLOv8, both using CIOU Loss. ang It is a newly added loss used to optimize the phase shift regression task.

[0155] The present invention relates to a multi-objective learning framework, including three loss functions, {L cls , L box , L ang}There are obvious differences between the losses corresponding to different detection tasks. Since all detection tasks rely on the same feature extractor, this may lead to conflicts between objectives. In order to overcome the difficulty of model parameter optimization caused by this multi-objective conflict, the present invention adopts a multi-gradient descent optimization algorithm based on Pareto optimization theory, which combines the Pareto genetic algorithm to optimize the total loss function. The algorithm first determines the weight of each loss term, and then generates a set of solutions, each of which corresponds to a different combination of weights, which constitute the Pareto frontier. On the Pareto frontier, an equilibrium point is selected to determine the weight, and then these loss terms are dynamically weighted to finally achieve the update and optimization of the model parameters.

[0156] S5, using the trained target detection model, predicts the position, size and rotation angle of each target, and obtains the final phase by predicting the cosine values ​​of multiple different phase shifts;

[0157] The core of the phase shift encoder in the target detection model can be summarized as the encoding function f(·) and the decoding function f -1 (·), the input and output of the decoding function are opposite to it. Its calculation method is as follows:

[0158]

[0159] In the formula, the input of the encoding function is angle θ, and the output is N step The cosine value Ps of the phase shift.

[0160] Based on the final phase, the non-maximum suppression method is applied to remove overlapping detection boxes and retain the best detection results.

[0161] The detection boxes are sorted according to their confidence scores, and then the box with the highest confidence score is selected as the reference, and its intersection over union (IoU) with all other boxes is calculated. If the IoU between other boxes and the reference box exceeds the preset threshold, they are considered to be detecting the same target, and these boxes are suppressed. This process is iterated until all detection boxes are processed, and the detection box with the highest confidence score and no suppression is output as the final result.

[0162] In the test phase, 0.7 is used as the threshold of NMS. For each detection result, the division of positive and negative samples is determined by whether the IoU score with the GT box is greater than the IoU threshold, and the correctness or error of the prediction is determined by whether the predicted category is consistent with the true category.

[0163] Based on similar inventive concepts, an embodiment of the present invention further provides a computer storage medium storing a readable program, which can execute the above-mentioned rotating target detection method based on phase shift encoding and directional feature enhancement when the program is running.

[0164] Based on similar inventive concepts, an embodiment of the present invention provides an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0165] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned rotating target detection method based on phase shift encoding and directional feature enhancement.

[0166] Based on similar inventive concepts, an embodiment of the present invention further provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to the above-mentioned rotating target detection method based on phase shift encoding and directional feature enhancement.

[0167] Example 2

[0168] In this embodiment, in order to verify the effectiveness of the target detection method of the present invention, a test verification is performed;

[0169] The verification process is:

[0170] 1. Dataset preparation: Select DOTA and HRSC2016 datasets, which contain objects of various directions and sizes and are suitable for evaluating rotation target detection algorithms.

[0171] 2. Preprocessing: Standardize the dataset, including image scaling, normalization, and data augmentation, to match the data processing flow in the training phase.

[0172] 3. Model loading: Load the trained YOLO-PO model, which contains key components such as the directional feature enhancement module and the phase shift encoder.

[0173] 4. Feature map visualization: In order to verify the effectiveness of the directional feature enhancement module, the feature maps of the baseline model and the YOLO-PO model are generated and visualized. The response strength of the two at the detection target is observed and compared.

[0174] 5. Target detection execution: Input the preprocessed test data into the YOLO-PO model, perform target detection, and collect the prediction results output by the model.

[0175] 6. Non-maximum suppression: Apply the non-maximum suppression (NMS) step to remove overlapping detection boxes and retain the best detection results.

[0176] 7. Performance evaluation: Use evaluation indicators such as mean average precision (mAP) to quantitatively evaluate the detection performance of the YOLO-PO model.

[0177] 8. Visualization of detection results: Display the comparison of detection results of the YOLO-PO model and the baseline model for the same image. Analyze and demonstrate the advantages of the YOLO-PO model in small object recall rate and rotation angle prediction accuracy.

[0178] 9. Ablation study: Conduct ablation experiments to evaluate the impact of the directional feature enhancement module and phase shift encoder on model performance. Combine the test results to analyze the performance of the YOLO-PO model in different scenarios and verify its effectiveness and superiority.

[0179] In order to more intuitively illustrate the effectiveness of the directional feature enhancement module, Figure 7 The difference between the feature map visualization results of the baseline model and the model using the directional feature enhancement module is compared in Figure 1. The upper layer is the feature map visualization result of the baseline model, and the lower layer is the feature map visualization result of the model using the directional feature enhancement module. Figure 7 It can be seen that compared with the baseline model, the feature map of the model using directional feature enhancement has a stronger response at the detection target, which obviously helps to improve the detection accuracy of the model.

[0180] In addition, in order to more intuitively illustrate the effectiveness and superiority of the present invention, Figure 8 The YOLO-PO model and the baseline model are compared in the detection results of the same image. Figure 8 It can be seen that the YOLO-PO model not only has a higher recall rate for small targets, but also predicts the rotation angle of the target bounding box more accurately.

[0181] Example 3

[0182] Based on the target detection method proposed in Example 1, in this embodiment, a rotating target detection system based on phase shift encoding and directional feature enhancement is proposed, which specifically includes:

[0183] Image data acquisition unit: acquires image data set;

[0184] Data enhancement unit: perform data enhancement on image datasets;

[0185] Model training unit: the data-enhanced image is input into the target detection model for training; the target detection model includes: adaptive rotation convolution network, directional feature enhancement module and phase shift encoder; the directional feature enhancement module includes: multi-scale attention module, active rotation filter and rotation invariant pooling layer;

[0186] Model optimization unit: Use Pareto optimization theory to adaptively weight each loss term, construct the total loss function of the target detection model, and update the target detection model parameters through a multi-gradient descent algorithm;

[0187] Phase prediction unit: Use the trained target detection model to predict the position, size and rotation angle of each target, and decode the cosine values ​​of multiple different phase shifts to obtain the final phase;

[0188] Detection result screening unit: Based on the final phase, the non-maximum suppression method is applied to remove overlapping detection frames and retain the best detection results.

[0189] The method of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CDROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0190] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A rotating target detection method based on phase shift encoding and directional feature enhancement, characterized in that: The following steps are involved: Get image dataset; Perform data augmentation on image datasets; The data-enhanced images are input into the target detection model for training; The target detection model includes: ARC network, EMA module, ARF, rotation invariant pooling layer and phase shift encoder; the data-enhanced image is input into the ARC network to extract the features of targets in any direction, the EMA module performs adaptive feature optimization on the features, the ARF encodes the optimized feature direction information, generates direction-sensitive features for bounding box regression, and the rotation invariant pooling layer extracts directional features from the direction-sensitive features for classification; the phase shift encoder linearly maps the angle of the rotating box to the phase of the corresponding frequency, predicts the phase shift cosine value, and decodes to obtain the final phase; Use Pareto optimization theory to adaptively weight each loss term, construct the total loss function of the target detection model, and update the target detection model parameters through the multi-gradient descent algorithm; The trained target detection model is used to predict the position, size and rotation angle of each target, and the final phase is obtained by decoding the cosine values ​​of multiple different phase shifts. Based on the final phase, the non-maximum suppression method is applied to remove overlapping detection boxes and retain the best detection results.

2. The rotating target detection method based on phase shift encoding and directional feature enhancement according to claim 1 is characterized in that: The ARC network includes an ARConv module and an ARC-C2f module; The ARConv module replaces the standard 2D convolution in the convolution module Conv with an adaptive rotation convolution, retaining its batch normalization layer and SiLU activation function layer; The ARC-C2f module replaces the 3×3 convolution module in the bottleneck network of the C2f module with an ARConv module with the same size convolution kernel, while retaining its input and output 1×1 convolution modules.

3. The rotating target detection method based on phase shift encoding and directional feature enhancement according to claim 1 is characterized in that: The EMA module uses a multi-scale attention structure composed of two parallel branches of convolutions with different receptive fields, and captures the pixel-level attention of the input feature map without reducing the channel dimension by feature grouping and cross-scale feature fusion; for any given input feature The multi-scale attention module divides X into G groups of sub-features according to the channel dimension. After the division, the input feature is expressed as: In the formula, GAP w and GAP h Represent the global average pooling on the horizontal and vertical axes respectively, F1 represents the 1×1 convolution function, F3 represents the 3×3 convolution function, and σ(·) is the sigmiod function; Output of 1×1 branch and the output of the 3×3 branch The manifestation is: Among them, GAP represents the global average pooling on the entire plane; The final output Y of the EMA module is i for:

4. The rotating target detection method based on phase shift encoding and directional feature enhancement according to claim 1 is characterized in that: The phase shift encoder includes a mapping process, an encoding process and a decoding process; The mapping process is: Where θ is the rotation angle of the bounding box, is the corresponding phase; The encoding process is to encode the phase into cosine values ​​of different phase shifts. The encoding formula is: Among them, x n It represents a series of cosine values ​​calculated during the encoding process, N step is the number of phase shift steps; The decoding process is the process of inversely deducing the phase from the cosine value of the multi-step phase shift. The calculation formula is:

5. The rotating target detection method based on phase shift encoding and directional feature enhancement according to claim 1 is characterized in that: The total loss function L of the target detection model is: L=ω1L cls +ω2L box +ω3L ang L ang =|f p ―f t | Among them, ω1, ω2, ω3 are hyperparameters for adjusting the loss weights of each task; L cls is the classification loss, is the binary cross entropy loss; L box is the bounding box coordinate regression loss L box , L ang is the angle loss for phase shift encoding; IOU is the intersection-over-union ratio between the predicted box and the real box, (x1, y1) and (x2, y2) are the coordinates of the center points of the predicted box and the real box respectively, d is the distance between the center points of the predicted box and the real box, w1, h1 and w2, h2 are the width and height of the predicted box and the real box respectively, and α is a hyperparameter for balancing the difference in aspect ratio; f t is the rotation angle θ t Encode to get; f p is the cosine of the predicted phase shift.

6. The rotating target detection method based on phase shift encoding and directional feature enhancement according to claim 1 is characterized in that: The process of applying the non-maximum suppression method to remove overlapping detection frames includes: Sort the detection frames according to their confidence scores, select the detection frame with the highest confidence as the reference frame, and calculate its intersection-over-union ratio with other detection frames; If the intersection-over-union ratio of other detection frames with the reference frame exceeds a preset threshold, they are considered to have detected the same target and these detection frames are suppressed. After iterations, until all detection frames are processed, the detection frame with the highest confidence and not suppressed is output as the final result.

7. A rotating target detection system based on phase shift encoding and directional feature enhancement, characterized in that: include: Image data acquisition unit: acquires image data set; Data enhancement unit: perform data enhancement on image datasets; Model training unit: The data-enhanced image is input into the target detection model for training; The target detection model includes: ARC network, EMA module, ARF, rotation invariant pooling layer and phase shift encoder; the data-enhanced image is input into the ARC network to extract the features of targets in any direction, the EMA module performs adaptive feature optimization on the features, the ARF encodes the optimized feature direction information, generates direction-sensitive features for bounding box regression, and the rotation invariant pooling layer extracts directional features from the direction-sensitive features for classification; the phase shift encoder linearly maps the angle of the rotating box to the phase of the corresponding frequency, predicts the phase shift cosine value, and decodes to obtain the final phase; Model optimization unit: Use Pareto optimization theory to adaptively weight each loss term, construct the total loss function of the target detection model, and update the target detection model parameters through a multi-gradient descent algorithm; Phase prediction unit: Use the trained target detection model to predict the position, size and rotation angle of each target, and decode the cosine values ​​of multiple different phase shifts to obtain the final phase; Detection result screening unit: Based on the final phase, the non-maximum suppression method is applied to remove overlapping detection frames and retain the best detection results.

8. A computer storage medium storing a readable program, characterized in that: When the program is running, it can execute the rotating target detection method based on phase shift encoding and directional feature enhancement as described in any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the rotating target detection method based on phase shift encoding and directional feature enhancement as described in any one of claims 1-6.

10. A computer program product comprising computer instructions, characterized in that: The computer instructions instruct the computing device to perform operations corresponding to the rotating target detection method based on phase shift encoding and directional feature enhancement as described in any one of claims 1-6.

Images