Sample optimization method, detection method and system

Abstract

The invention belongs to the technical field of remote sensing ship detection, and particularly discloses a sample optimization method, a detection method and a system, and the sample optimization method comprises the following steps: S1, employing a shape perception quality evaluation strategy, and dynamically adjusting an IoU threshold value through the shape information of a target to select a training sample; and S2, endowing the selected positive sample with quality information by adopting a shape center distance weighting mode, and optimizing the selection of the training sample. By adopting the technical scheme, the I oU threshold is dynamically adjusted through the shape information of the target to select the training sample, and the selected positive sample is endowed with the quality information by adopting a shape center distance weighting mode, so that the selection process of the training sample is optimized, the sample quality is optimized, and the detection precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing ship detection, and relates to a sample optimization method, a detection method and a system. Background Art

[0002] Target detection in remote sensing images has received extensive attention in recent years. Especially in the fields of marine monitoring, ship management, maritime safety, etc., ship detection has become an important research topic. Compared with target detection in natural scenes, remote sensing ship detection faces many challenges, including:

[0003] 1) Large variation in target scale: Remote sensing images cover a large range of sea areas, and the scale of ship targets varies significantly. From small fishing boats to large cargo ships, the target sizes differ greatly, posing a great challenge to the detection model.

[0004] 2) Elongated target shape and variable directions: Ships usually present an elongated shape with a high aspect ratio and may be distributed in any direction in remote sensing images. Traditional detection methods based on horizontal bounding boxes are difficult to accurately depict the target boundary and are prone to introducing a large amount of background interference.

[0005] 3) Dense distribution of targets and easy overlap: In specific scenes such as ports and waterways, ship targets often show dense arrangements, and there is partial occlusion or overlap between targets, increasing the difficulty of correct separation and detection.

[0006] To address the above challenges, recent research has mainly focused on rotation target detection methods, enabling the detection bounding box to rotate freely to more accurately enclose the target. However, when facing ship targets with a high aspect ratio, the performance of such methods is often unsatisfactory. The main reason is that existing models are difficult to effectively capture the slender structural features of ships, resulting in unstable rotation angle prediction and inaccurate capture of the target boundary contour, thus affecting the detection accuracy. Summary of the Invention

[0007] The purpose of the present invention is to provide a sample optimization method, a detection method and a system, which optimize the sample quality and improve the detection accuracy.

[0008] To achieve the above purpose, the basic solution of the present invention is: A sample optimization method, comprising the following steps:

[0009] S1, using a shape-aware quality assessment strategy, dynamically adjusting the IoU threshold through the shape information of the target to select training samples;

[0010] S2, adopting a method of weighting by the distance from the shape center, assigning quality information to the selected positive samples, and optimizing the selection of training samples.

[0011] The working principle and beneficial effects of this basic solution are as follows: This technical solution adaptively adjusts the IoU threshold according to the shape information of the target to achieve dynamic sample selection. By introducing the weighted shape center distance to distinguish the quality of samples. The smaller the distance value, the closer the sample point is to the target center, and the higher its quality score. Therefore, this distance value can be used to more accurately evaluate the quality of each positive sample, thereby optimizing the sample selection process.

[0012] Further, step S1 is specifically as follows:

[0013] Adopt a shape-aware quality assessment strategy, and dynamically adjust the IoU threshold through the shape information of the target to select training samples, specifically as follows:

[0014] Adaptive adjustment of the IoU threshold according to the shape information of the target to achieve dynamic sample selection:

[0015]

[0016] Among them, represents the IoU threshold for dynamically selecting samples, μ represents the mean of the IoU values of candidate samples; σ represents the standard deviation of the IoU values of candidate samples; ω represents the weighting parameter, which is used to control the influence degree of the aspect ratio on the weight factor; SI represents the shape information of the target, and w and h represent the width and height of the ground truth box.

[0017] Dynamically adjusting the IoU threshold through the shape information of the target to select training samples is conducive to use.

[0018] Further, adopt the method of weighted shape center distance to assign quality information to the selected positive samples and optimize the sample selection process of training samples as follows:

[0019] Use the Euclidean distance from the sample point to the object center and the shape information of the object to calculate the weighted shape center distance value

[0020]

[0021] Among them, (x i , y i ) represents the center coordinates of the true bounding box, (x j , y j ) represents the coordinates of the sample point, h i and w i are the height and width of the true bounding box respectively;

[0022] Calculate the quality score Q of the positive sample ij :

[0023]

[0024] Adopt the method of weighting by the distance from the shape center to assign quality information to the selected positive samples, thereby optimizing the selection process of training samples.

[0025] The present invention also provides a detection method based on the sample optimization method of the present invention, including the following steps:

[0026] S41. Embed the dynamic refinement rotation convolution module (DRRCM) into the backbone network ResNet50 of the object detection model, replace some of its 3×3 convolution modules, and DRRCM uses the data augmentation spatial attention module (DESAM) to predict the weights and angles of the rotation convolution kernels;

[0027] Combine the predicted weights and angle parameters of the rotation convolution kernels, and use the dynamic refinement rotation convolution module (DRRCM) to adaptively adjust the convolution kernels according to the pose information of the oriented objects to generate direction-sensitive feature maps, so that they are accurately aligned with the object features;

[0028] S42. Use the anchor refinement feature alignment module (ARFAM) to generate corrected predicted anchor boxes on the direction-sensitive feature maps based on the regression branch, as a guide to dynamically adjust the positions of the feature sampling points;

[0029] S43. Use the shape-aware quality assessment strategy to dynamically adjust the IoU threshold through the shape information of the objects to select training samples;

[0030] Adopt the method of weighting by the distance from the shape center to assign quality information to the selected positive samples, and optimize the selection process of training samples;

[0031] S44. Collect the remote sensing images to be detected and input them into the optimized object detection model to obtain the object detection results.

[0032] This technical solution designs a dynamic refinement rotation convolution module (DRRCM), uses the data augmentation spatial attention module (DESAM) to predict the dynamic rotation convolution kernel parameters, and realizes the feature alignment with adaptive rotation angles of the objects.

[0033] Construct an anchor refinement feature alignment module (ARFAM), guide the feature sampling positions by regression to correct the preset anchor boxes, and solve the problem of feature-space misalignment. Propose a shape-aware quality assessment (SAQA) strategy, dynamically adjust the IoU threshold based on the object shape information (SI), and optimize the sample matching quality by combining the distance weighting from the shape center.

[0034] Improve the accuracy and robustness of remote sensing ship detection by combining feature representation optimization and dynamic matching mechanisms.

[0035] Furthermore, the method for DRRCM to use the Data Enhancement Spatial Attention Module (DESAM) to predict the weights and angles of rotation convolution kernels is as follows:

[0036] Perform channel-based average pooling and max pooling on the feature map F of the original image after depth convolution, denoted as P avg (.), and P max (.), to extract the spatial relationship:

[0037] S avg = P avg (F), S max = P max (F)

[0038] where S avg and S max are the spatial feature descriptors after average pooling and max pooling. To allow information interaction between different spatial descriptors, the features of spatial pooling are concatenated, and a convolutional layer is used to convert the pooled features into C in spatial attention maps S':

[0039]

[0040] where C in represents the number of input channels; for each spatial attention map S', a sigmoid activation function is applied to obtain the individual spatial mask S i ' of each convolution kernel:

[0041] S i ' = sigmoid(S')

[0042] Use the spatial mask to weight the features after depth convolution, and compress them into a C in -dimensional feature vector V Cin :

[0043] V Cin = P avg (S i ' × F)

[0044] Send the pooled feature vectors into two branches respectively. The first is the rotation kernel angle prediction branch. Input the feature vector into this branch, and after passing through Dropout, a linear layer, Softsign activation, and multiplying by a scale factor, a set of angles θ i are obtained:

[0045] θ i = K(Softsign(z θ ))

[0046] Among them, z θ The bias is not set for the linear layer to ensure that the angle prediction only depends on the change of the input features and avoid learning a biased angle; is a scaling factor to expand the rotation range, and the proportion parameter is used to adjust the angle range;

[0047] The second is the rotation kernel weight prediction branch. The feature vector is input into this branch and passes through Dropout, a linear layer, and the Sigmoid activation to obtain a set of weights λ i :

[0048] λ i = sigmoid(z a )

[0049] Among them, z α The bias is set for the linear layer to improve the flexibility of the model.

[0050] The weighted fused features are average pooled and input into the kernel angle prediction branch and the kernel weight prediction branch. This module can make the network more accurately focus on the key feature positions in the rotated object detection, so as to accurately generate the weights and angles of the predicted rotation kernel.

[0051] Furthermore, the rotation angle θ i predicted by DESAM is used to reparameterize the weights inside the convolution kernel, so that the convolution kernel W i can be dynamically adjusted according to different input feature maps to achieve adaptive rotation:

[0052] Y i ' = rotate(Y i , -θ i )

[0053] W i ' = interpolation(W i , Y i ')[[]]END]]

[0054] Among them, Y i represents the coordinates of the original sampling points, and Y i ' represents the new sampling point coordinates after rotating the original sampling points Y i counterclockwise by the angle θ i to achieve alignment between the convolution kernel and the features; rotate represents the rotation operation; W i ' represents the reparameterized convolution kernel; interpolation(.) represents bilinear interpolation, which is used to calculate the weight value at the new position after the convolution kernel W i is rotated;

[0055] Multiply the reparameterized convolution kernel with the corresponding λ i weight, sum them up, and then perform a convolution operation with the input feature map to finally generate high-quality orientation-aware feature Y:

[0056]

[0057] where n represents the number of rotation convolution kernels.

[0058] Obtain high-quality features, which is beneficial for subsequent use.

[0059] Furthermore, through the Anchor Refinement Feature Alignment Module (ARFAM), based on the regression branch, quickly generate corrected predicted anchor boxes on the orientation-sensitive feature map as a guide to dynamically adjust the positions of feature sampling points. The specific steps are as follows:

[0060] The features output by DRRCM are fused through FPN to obtain a feature map, and only one initial square anchor point is preset at each position of this feature map. Through the regression branch, it is corrected into a high-quality directional anchor point, and the offset of the predicted anchor box regression target is predicted:

[0061]

[0062] where (x, y, w, h, θ) represent the center coordinates, width, height, and angle parameters of the initial anchor point; (x g , y g , w g , h g , θ g ) represent the center coordinates, width, height, and angle parameters of the true bounding box; (Δx g , Δy g , Δw g , Δh g , Δθ g ) represent the offsets between the true bounding box and the initial anchor point. By regressing these offsets, the model can adjust the initial anchor point to a corrected predicted anchor box closer to the true bounding box; R(θ) represents the rotation transformation matrix used to convert the center point coordinates of the true bounding box to the coordinate system relative to the initial anchor point; k represents the proportionality coefficient used to adjust the angle value to ensure that the rotation angle is within a reasonable range;

[0063] Use the corrected predicted anchor box as a guide to adjust the positions of feature sampling points to achieve dynamic convolution alignment. On the basis of the original sampling points of the standard convolution, add an offset o deduced from the corrected predicted anchor box:

[0064]

[0065] where Indicates the position of the sampling point for the predicted anchor box; (p0 + p n ) indicates the position of the regular sampling point for the standard convolution; p0 and p n respectively represent the two-dimensional coordinates and relative offsets of the sampling point, and R represents the regular grid of the standard convolution [(p x , p y )];

[0066] The dynamic alignment convolution combines the offset o and the input feature x, enabling the sampling point position to be adjusted according to the shape and orientation of the predicted anchor box to match the geometric characteristics of the actual target:

[0067]

[0068] Among them, Y(p) represents the value of the output feature map at position p, w(p n ) is the weight of the convolution kernel at position p[[ID=:20]] n , and x(·) represents the input feature map after feature fusion.

[0069] Based on the high-quality feature map generated by DRRCM in the backbone network, ARFAM further refines the anchor points through regression operations, calculates the offset field according to the refined anchor point parameters, realizes dynamic adjustment of the position of the alignment convolution sampling points, and generates a feature representation that is more accurately aligned with the target object.

[0070] The present invention also provides a sample optimization system, including a processing module, and the processing module executes the sample optimization method of the present invention to complete sample optimization.

[0071] This system accurately evaluates the quality of each positive sample, thereby optimizing the sample selection process.

[0072] The present invention also provides a detection system, including a processing unit, the processing unit includes a dynamic refinement rotation convolution module DRRCM and an anchor point refinement feature alignment module ARFAM, and the processing unit executes the detection method of the present invention to obtain the remote sensing image target detection result.

[0073] This system effectively improves the target feature alignment ability and the adaptability of the label assignment strategy, can more accurately capture the rotation features of high aspect ratio ships, and significantly improves the detection accuracy and robustness. Brief Description of the Drawings

[0074] Figure 1 is a schematic flow chart of the sample optimization method of the present invention;

[0075] Figure 2 is a schematic flow chart of the detection method of the present invention;

[0076] Figure 3It is a schematic structural diagram of the DESAM module of the detection system of the present invention;

[0077] Figure 4 It is a schematic structural diagram of the DRRCM module of the detection system of the present invention;

[0078] Figure 5 It is a schematic structural diagram of the Anchor Refinement Feature Alignment Module ARFAM of the detection system of the present invention;

[0079] Figure 6 It is a schematic diagram of the IoU of the aircraft in the sample optimization method of a preferred embodiment of the present invention;

[0080] Figure 7 It is a schematic diagram of the IoU of the ship in the sample optimization method of another preferred embodiment of the present invention;

[0081] Figure 8 It is a heatmap visualization comparison diagram of the detection method in a preferred embodiment of the present invention;

[0082] Figure 9 It is a partial visualization detection result diagram obtained by RDAF-Net of the detection method in a preferred embodiment of the present invention. Detailed implementation manners

[0083] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the drawings are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0084] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.

[0085] In the description of the present invention, unless otherwise specified and defined, it should be noted that the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the communication inside two elements. It can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific situations.

[0086] The present invention discloses a sample optimization method, asFigure 1 As shown in the figure, it includes the following steps:

[0087] S1. Using the shape-aware quality assessment strategy, dynamically adjust the IoU threshold through the shape information of the target to select training samples;

[0088] S2. Adopt the method of weighting by the distance from the shape center, assign quality information to the selected positive samples, and optimize the selection of training samples.

[0089] In a preferred embodiment of the present invention, step S1 is specifically as follows:

[0090] Adopting the shape-aware quality assessment strategy, dynamically adjust the IoU threshold through the shape information of the target to select training samples, specifically as follows:

[0091] Adaptively adjust the IoU threshold according to the shape information of the target to achieve dynamic sample selection:

[0092]

[0093] wherein, represents the IoU threshold for dynamically selecting samples, μ represents the mean value of the IoU values of candidate samples; σ represents the standard deviation of the IoU values of candidate samples; ω represents a weighting parameter used to control the influence degree of the aspect ratio on the weight factor; SI represents the shape information of the target, and w and h represent the width and height of the ground truth box.

[0094] In a preferred embodiment of the present invention, the selected positive samples are optimized by introducing the method of weighting by the distance from the shape center, evaluating their positions and adding quality information. The method of adopting the method of weighting by the distance from the shape center to assign quality information to the selected positive samples and optimize the selection process of training samples is as follows: [[ID=३१]]

[0095] Use the Euclidean distance from the sample point to the object center and the shape information of the object to calculate the shape center distance weighting value

[0096]

[0097] wherein, (x i , y i ) represents the center coordinates of the true bounding box, (x j , y j ) represents the coordinates of the sample point, h i and w i are the height and width of the true bounding box respectively;

[0098] Calculate the quality score Q ij of the positive sample:

[0099]

[0100] By introducing the weighted distance of the shape center to distinguish the quality of samples, the smaller the distance value, the closer the sample point is to the target center, and the higher its quality score. Therefore, using this distance value can more accurately evaluate the quality of each positive sample, thereby optimizing the sample selection process.

[0101] The present invention uses a non-linear function. For targets with a large difference in aspect ratio, the sample selection dynamically adjusts the IoU threshold according to the shape information of the sample, such as Figure 6 and Figure 7 , where the green box represents the ground truth box, and the burgundy box and yellow box represent the predicted boxes rotated by 15° and 30°. The overlapping areas of the burgundy and yellow colors represent the overlapping areas of the predicted box and the ground truth box, that is, IoU. It can be clearly observed that the vessels with high aspect ratio are much more affected by the angle than the aircraft with regular shapes.

[0102] The present invention also provides a detection method based on the sample optimization method described in the present invention. Based on the single-stage rotation double alignment feature network (RDAF-Net), by combining feature characterization optimization and dynamic matching mechanism, the accuracy and robustness of remote sensing ship detection are improved. As Figure 2 shown, the detection method includes the following steps:

[0103] S41. In order to avoid the fixed convolution method of standard convolution, enhance the characterization ability of the target, and thus achieve accurate target detection. The dynamic refinement rotation convolution module DRRCM is embedded into the backbone network ResNet50 of the target detection model, replacing some of its 3×3 convolution modules. DRRCM uses the data augmentation spatial attention module DESAM to predict the weights and angles of the rotation convolution kernel; design the dynamic refinement rotation convolution module (DRRCM), use the data augmentation spatial attention module (DESAM) to predict the dynamic rotation convolution kernel parameters, and realize the feature alignment of the target rotation angle adaptively.

[0104] Combine the predicted weights and angle parameters of the rotation convolution kernel, and use the dynamic refinement rotation convolution module DRRCM to adaptively adjust the convolution kernel according to the attitude information of the oriented target to generate a direction-sensitive feature map, so that it is accurately aligned with the target feature;

[0105] S42. Use the anchor refinement feature alignment module ARFAM to generate corrected predicted anchor boxes on the direction-sensitive feature map based on the regression branch, as a guide to dynamically adjust the positions of the feature sampling points; construct the anchor refinement feature alignment module (ARFAM), and solve the feature-space misalignment problem by regressing and correcting the preset anchor boxes to guide the feature sampling positions.

[0106] S43. Using the shape-aware quality assessment strategy, the IoU threshold is dynamically adjusted based on the shape information of the target to select training samples;

[0107] Adopting the method of weighting by the distance from the shape center, quality information is assigned to the selected positive samples to optimize the selection process of training samples; A shape-aware quality assessment (SAQA) strategy is proposed, which dynamically adjusts the IoU threshold based on the target shape information (SI) and optimizes the sample matching quality by combining the weighting of the distance from the shape center.

[0108] S44. Collect the remote sensing image to be detected and input it into the optimized object detection model to obtain the object detection result.

[0109] Through the collaborative effect of the above three modules, RDAF-Net effectively improves the object feature alignment ability and the adaptability of the label assignment strategy in the ship detection task, can capture the rotation features of high aspect ratio ships more accurately, and significantly improves the detection accuracy and robustness. The experimental results show that RDAF-Net has achieved excellent performance on multiple remote sensing datasets, demonstrating its application potential in complex marine scenarios.

[0110] In a preferred embodiment of the present invention, the method for DRRCM to use the data augmentation spatial attention module DESAM to predict the weights and angles of the rotation convolution kernels is as follows:

[0111] Performing channel-based average pooling and max pooling on the feature map F of the original image after depth convolution, denoted as P avg (.), and P max (.), to extract the spatial relationship:

[0112] S avg =P avg (F), S max =P max (F)

[0113] where S avg and S max are the spatial feature descriptors after average pooling and max pooling. To allow information interaction between different spatial descriptors, the spatially pooled features are concatenated and a convolutional layer is used to convert the pooled features (with 2 channels) into C in spatial attention maps S':

[0114]

[0115] where C in represents the number of input channels; for each spatial attention map S', a sigmoid activation function is applied to obtain the individual spatial mask S i':

[0116] S i ' = sigmoid(S'),

[0117] The features after depth convolution are weighted using a spatial mask and compressed into a C in -dimensional feature vector V Cin :

[0118] V Cin = P avg (S i ' × F),

[0119] The pooled feature vector is sent to two branches respectively. The first is the rotation kernel angle prediction branch. The feature vector is input into this branch and passes through Dropout, a linear layer, Softsign activation, and multiplication by a scale factor to obtain a set of angles θ i :

[0120] θ i = K(Softsign(z θ ))

[0121] where z θ has no bias set in the linear layer to ensure that the angle prediction only depends on the changes in the input features and avoid learning offset angles; is the scale factor to expand the rotation range, and the proportion parameter is used to adjust the range of the angle;

[0122] The second is the rotation kernel weight prediction branch. The feature vector is input into this branch and passes through Dropout, a linear layer, and Sigmoid activation to obtain a set of weights λ i :

[0123] λ i = sigmoid(z a )

[0124] where z α has a bias set in the linear layer to improve the flexibility of the model.

[0125] In a preferred embodiment of the present invention, DESAM is initialized from a truncated normal distribution with a mean of zero and a standard deviation of 0.2 to help the model converge faster and reduce the instability in the initial stage of training. The rotation angle θ i predicted by DESAM is used to reparameterize the weights inside the convolution kernel, enabling the convolution kernel W i to be dynamically adjusted according to different input feature maps to achieve adaptive rotation:

[0126] Y iY' = rotate(Y i , -θ i )

[0127] W i W' = interpolation(W i , Y i ')

[0128] Among them, Y i represents the coordinates of the original sampling points, and Y i ' represents the new sampling point coordinates after rotating the original sampling point Y i counterclockwise by the angle θ i to achieve the alignment of the convolution kernel and the features; rotate represents the rotation operation; W i ' represents the reparameterized convolution kernel; interpolation(.) represents bilinear interpolation, which is used to calculate the weight value at the new position of the rotated W i convolution kernel;

[0129] Multiply the reparameterized convolution kernel by the corresponding λ i weight and sum, and then perform a convolution operation with the input feature map to finally generate a high-quality orientation-aware feature Y:

[0130]

[0131] Among them, n represents the number of rotated convolution kernels.

[0132] In a preferred embodiment of the present invention, based on the high-quality feature map generated by DRRCM in the backbone network, ARFAM is used to further refine the anchor points through a regression operation, and the offset field is calculated according to the refined anchor point parameters to dynamically adjust the position of the aligned convolution sampling points and generate a more accurate feature representation aligned with the target object.

[0133] Through the anchor point refinement feature alignment module ARFAM, based on the regression branch, corrected predicted anchor boxes are quickly generated on the direction-sensitive feature map as a guide to dynamically adjust the position of the feature sampling points. The specific steps are as follows:

[0134] The features output by DRRCM are fused through FPN to obtain a feature map, and only one initial square anchor point is preset at each position of the feature map, which is corrected to a high-quality directional anchor point through the regression branch, thereby reducing the large number of anchor points preset on the feature map to reduce the computational amount. Predict the offset of the regression target of the anchor box:

[0135]

[0136] Among them, (x, y, w, h, θ) represent the center coordinates, width, height, and angle parameters of the initial anchor point; (xg , y g , w g , h g , θ g ) represent the center coordinates, width, height, and angle parameters of the true bounding box; (Δx g , Δy g , Δw g , Δh g , Δθ g ) represent the offsets between the true bounding box and the initial anchor point. By regressing these offsets, the model can adjust the initial anchor point to a corrected predicted anchor box closer to the true bounding box; R(θ) represents the rotation transformation matrix used to convert the center point coordinates of the true bounding box into the coordinate system relative to the initial anchor point; k represents the proportionality coefficient used to adjust the angle value to ensure that the rotation angle is within a reasonable range;

[0137] To achieve feature extraction of the oriented target, the position of the feature sampling point is adjusted through the corrected predicted anchor box as a guide to achieve dynamic convolutional alignment. Based on the original sampling points of the standard convolution, an offset o deduced from the corrected predicted anchor box is added:

[0138]

[0139] Among them, represents the position of the sampling point of the predicted anchor box; (p0 + p n ) represents the position of the regular sampling point of the standard convolution; p0 and p n represent the two-dimensional coordinates and relative offset of the sampling point respectively, and R represents the regular grid [(p x , p y )] of the standard convolution;

[0140] The dynamic alignment convolution combines the offset o and the input feature x, enabling the sampling point position to be adjusted according to the shape and direction of the predicted anchor box, thus better matching the geometric characteristics of the actual target:

[0141]

[0142] Among them, Y(p) represents the value of the output feature map at position p, w(p n ) is the weight of the convolution kernel at position p n , and x(·) represents the input feature map after feature fusion.

[0143] The present invention also provides a sample optimization system, including a processing module, and the processing module executes the sample optimization method of the present invention to complete sample optimization.

[0144] This system accurately evaluates the quality of each positive sample, thereby optimizing the sample selection process.

[0145] To analyze the impact of different components designed in RDAF-Net, verification was carried out in a controlled variable manner on the HRSC2016 dataset. All experiments used the same settings to ensure the rigor of the experiments. The experimental results of the components are shown in Table 1:

[0146] Table 1. Impact of Each Component of RDAF-Net

[0147]

[0148] Using only DRRCM, the detection performance increased by 1.79% compared to the baseline, indicating that DRRCM can more effectively extract high-quality feature representations, thereby achieving accurate detection performance. When using DRRCM and ARFAM simultaneously, the detection effect reached an mAP of 84.89%, further increasing by 9.67%. This shows that even if only one anchor point is preset, through the alignment of convolutional features guided by the corrected high-quality anchor points, high-quality feature representations can be efficiently extracted while optimizing the anchor points. This indicates that these two methods not only do not conflict but also extract high-quality feature representations in a better progressive manner, effectively improving the detection performance.

[0149] When using the three components DRRCM, ARFAM, and SAQA simultaneously, the detection effect increased by 4.69% again. It can be clearly seen that in the label assignment process, SAQA makes reasonable sample selection based on the extracted high-quality feature maps and the optimized anchor boxes, and finally reaches an mAP of 89.58%, which is 17.94% higher than the baseline (RetinaNet).

[0150] A comparative experiment was carried out based on the HRSC2016 dataset (as shown in Table 2). The HRSC2016 dataset is a publicly available dataset for ship detection in optical remote sensing images released in 2016. This dataset collects images of 6 iconic ports from Google Earth and only contains two scenarios: offshore ships and nearshore ships. This dataset has a total of 1061 images (2976 object instances), the image spatial resolution is between 0.4 - 2m, and the pixel distribution is between 300×300 pixels and 1500×900 pixels. In our experiment, all image sizes were adjusted to 512×800. A total of 617 images were used for model training, among which the training set and the validation set were 436 images and 181 images respectively, and the remaining 444 images were used for testing.

[0151] Table 2. Comparative Experiment of HRSC2016

[0152] Methods Backbone Size Anchor mAP R2CNN R-101 800×800 21 73.07 RRPN R-101 800×800 54 79.08 RRD VGG16 384×384 13 84.30 RoI-Trans. R-101 512×800 5 86.20 DAL R-101 416×416 3 88.95 R-RetinaNet R-101 800×800 121 89.18 R3Det R-101 800×800 126 89.26 CFC-Net R-101 800×800 1 89.50 RDAF-Net R-50-DRRCM 512×800 1 89.58 RDAF-Net(aug) R-50-DRRCM 512×800 1 90.03 RDAF-Net(aug) R-101-DRRCM 512×800 1 90.05

[0153] Note: ('aug' represents data augmentation, and Anchor represents the number of anchor points preset at each point position on the feature map.)

[0154] As Figure 8 and Figure 9 shown, Figure 8 the first row of the pictures in [[ ]] is RetinaNet, and the second row is the RDAF-Net of the present invention.

[0155] Results of HRSC2016: The HRSC2016 dataset contains a large number of rotated ship images with high aspect ratios, various scales, and arbitrary orientations, which can fully verify the detection performance of our model for high-aspect-ratio oriented targets. The method of the present invention has achieved competitive performance on the HRSC2016 dataset.

[0156] When using R-101-DRRCM as the backbone network and adjusting the input image to 512×800 pixels, the method of the present invention reaches the highest mean average precision (mAP) of 90.05%. Even when using the lighter R-50-DRRCM, the method of the present invention can still achieve an mAP of 89.58%.

[0157] The method of the present invention only uses one square anchor point at each position of the feature map, but still outperforms the frameworks that preset a large number of rotated anchor points at each point position on the feature map. For example, 21 anchor points are preset in R2CNN and 126 anchor points are preset in R3Det. Compared with their best detection results, only using 1 anchor point and without data augmentation strategy for training and testing respectively improves by 16.51% and 0.32%. These results indicate that there is no need to preset a large number of rotated anchor boxes with different scales for oriented target detection. More importantly, high-quality basic feature extraction and optimization of high-quality predicted anchor boxes, and on this basis, reasonable sample training is selected simultaneously for target recognition.

[0158] The present invention dynamically adjusts the IoU according to the shape information of the object to select samples. IoU is the overlapping area between the ground truth box and the predicted box. During the process of sample selection, the fixed IoU is generally set to 0.5. Samples with a value higher than this are positive samples, and vice versa. The regression effect of the predicted box is affected by the shape of the samples. For targets with a large difference in aspect ratio, the regression effect of the fixed IoU is relatively poor (see Figure 6 and Figure 7 shown, the green is the ground truth box, the red and yellow are the predicted boxes rotated by 15° and 30°, respectively. The overlapping areas of the wine red and yellow represent the overlapping area between the predicted box and the ground truth box, that is, IoU. It can be clearly seen that Figure 7The height-width ratio target is greatly affected by the angle. Because the greater the deviation of the angle, the less color is covered. If a fixed IoU is used, the overlapping area between the two bounding boxes will be much smaller than the preset fixed IoU, resulting in filtering out a large number of targets with high height-width ratio, thus leading to sample imbalance. This patent application dynamically adjusts the IoU threshold according to the shape information of the samples. For Figure 6 regular targets, 0.5 can be used as the boundary between positive and negative samples. For Figure 7 targets with high height-width ratio, it is necessary to judge according to the shape information and dynamically adjust the IoU threshold to 0.4, 0.3, etc., which can ensure the balance of training samples.

[0159] The present invention also provides a detection system, including a processing unit. The processing unit includes a dynamic refinement rotation convolution module DRRCM and an anchor refinement feature alignment module ARFAM. The processing unit executes the detection method of the present invention to obtain the remote sensing image target detection result.

[0160] As Figure 3 shown, the data enhancement spatial attention module DESAM includes a depth convolution layer, a max pooling layer, a first average pooling layer, a concatenation (concat) layer, a convolution layer, a second average pooling layer, and two fully connected layers.

[0161] The output end of the depth convolution layer is respectively connected to the max pooling layer and the first average pooling layer. The output ends of the max pooling layer and the first average pooling layer are connected to the concat (concatenation) layer. The concat layer, the convolution layer, the second average pooling layer, and the two fully connected layers are connected in sequence.

[0162] As Figure 4 shown, the dynamic refinement rotation convolution module DRRCM includes a data enhancement spatial attention module DESAM and a convolution kernel that can be adaptively adjusted according to the pose information of the oriented target.

[0163] As Figure 5 shown, the anchor refinement feature alignment module ARFAM includes a regression branch layer and an alignment convolution layer connected in sequence.

[0164] This system effectively improves the target feature alignment ability and the adaptability of the label assignment strategy in the ship detection task, can more accurately capture the rotation features of ships with high height-width ratio, and greatly improves the detection accuracy and performance.

[0165] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc., mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0166] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A sample optimization method, characterized in that, It includes the following steps: S1. Using a shape-aware quality assessment strategy, dynamically adjust the IoU threshold through the shape information of the target to select training samples; S2. Adopt the method of weighting by the distance from the shape center to assign quality information to the selected positive samples and optimize the selection of training samples.

2. The sample optimization method according to claim 1, characterized in that Specifically, step S1 is as follows: Adopt a shape-aware quality assessment strategy and dynamically adjust the IoU threshold through the shape information of the target to select training samples, specifically: Adaptive adjust the IoU threshold according to the shape information of the target to achieve dynamic sample selection: Among them, represents the IoU threshold for dynamically selecting samples, μ represents the mean of the IoU values of the candidate samples; σ represents the standard deviation of the IoU values of the candidate samples; ω represents the weighting parameter, which is used to control the influence degree of the aspect ratio on the weight factor; SI represents the shape information of the target, and w and h represent the width and height of the ground truth box.

3. The sample optimization method according to claim 2, characterized in that, The method of adopting the method of weighting by the distance from the shape center to assign quality information to the selected positive samples and optimize the selection process of training samples is: Calculate the shape center distance weighted value by using the Euclidean distance from the sample point to the object center and the shape information of the object Among them, (x i , y i ) represents the center coordinates of the true bounding box, and (x j , y j ) represents the coordinates of the sample point. h i and w i are the height and width of the true bounding box respectively; Calculate the mass fraction Q of the positive samples ij :

4. A detection method based on the sample optimization method according to any one of claims 1-3, characterized in that, It includes the following steps: S41. Embed the dynamic refinement rotation convolution module DRRCM into the backbone network ResNet50 of the object detection model, replace some of its 3×3 convolution modules, and DRRCM uses the data augmentation spatial attention module DESAM to predict the weights and angles of the rotation convolution kernels; Combine the predicted weights and angle parameters of the rotation convolution kernels, and use the dynamic refinement rotation convolution module DRRCM to adaptively adjust the convolution kernels according to the pose information of the oriented target to generate a direction-sensitive feature map, so that it is accurately aligned with the target features; S42. Use the anchor refinement feature alignment module ARFAM to generate corrected predicted anchor boxes on the direction-sensitive feature map based on the regression branch as a guide to dynamically adjust the positions of the feature sampling points; S43. Using a shape-aware quality assessment strategy, dynamically adjust the IoU threshold through the shape information of the target to select training samples; Adopt the method of weighting by the distance from the shape center to assign quality information to the selected positive samples and optimize the selection process of training samples; S44. Collect the remote sensing image to be detected and input it into the optimized object detection model to obtain the object detection result.

5. The detection method according to claim 4, characterized in that The method for DRRCM to use the data augmentation spatial attention module DESAM to predict the weights and angles of the rotation convolution kernels is: Perform channel-based average pooling and max pooling on the feature map F of the original image after deep convolution, denoted as P avg (.) and P max (.), and extract the spatial relationship: S avg = P avg (F), S max = P max (F) Among them, S avg and S max are spatial feature descriptors after average pooling and max pooling. To allow information interaction between different spatial descriptors, the features of spatial pooling are concatenated, and a convolutional layer is used to convert the pooled features into C in spatial attention maps S': Among them, C in represents the number of input channels; for each spatial attention map S′, a sigmoid activation function is applied to obtain the individual spatial mask S′ of each convolutional kernel i : S′ i = sigmoid(S′), The features after depth convolution are weighted using a spatial mask and compressed into a feature vector V of dimension C through a global average pooling in -dimensional Cin : V Cin = P avg (S′ i × F), The pooled feature vectors are fed into two branches respectively. The first is the rotation kernel angle prediction branch. The feature vectors are input into this branch and pass through Dropout, a linear layer, Softsign activation, and multiplication by a scale factor to obtain a set of angles θ i : θ i = K(Softsign(z θ )) Among them, z θ No bias is set for the linear layer to ensure that the angle prediction only depends on the change of the input features and avoid learning a biased angle; is a scale factor to expand the rotation range, and the proportion parameter is used to adjust the range of the angle; The second is the rotation kernel weight prediction branch. The feature vector is input into this branch and passes through Dropout, a linear layer, and Sigmoid activation to obtain a set of weights λ i : λ i = sigmoid(z a ) Among them, z α The linear layer is provided with a bias to improve the flexibility of the model.

6. The detection method according to claim 5, characterized in that The rotation angle θ generated by DESAM prediction i The parameters reparameterize the weights inside the convolutional kernel, enabling the convolutional kernel W i to be dynamically adjusted according to different input feature maps, achieving adaptive rotation: Y i ′ = rotate(Y i , -θ i ) W i ′ = interpolation(W i , Y i ′) Among them, Y i represents the coordinates of the original sampling point, and Y i ' represents the new sampling point coordinates after rotating the original sampling point Y i counterclockwise by the angle θ i to achieve the alignment of the convolution kernel and the feature; rotate represents the rotation operation; W i ' represents the convolution kernel after reparameterization; interpolation(.) represents bilinear interpolation, which is used to calculate the weight value of W i at the new position after the convolution kernel is rotated; Multiply the reparameterized convolution kernel by the corresponding λ i weight, sum them up, and then perform a convolution operation with the input feature map to finally generate high-quality orientation-aware features Y: Where n represents the number of rotation convolution kernels.

7. The detection method according to claim 4, characterized in that, Through the anchor refinement feature alignment module ARFAM, quickly generate corrected predicted anchor boxes on the direction-sensitive feature map based on the regression branch as a guide to dynamically adjust the positions of the feature sampling points. The specific steps are: The features output by DRRCM are fused through FPN features to obtain a feature map, and only one initial square anchor point is preset at each position of the feature map, and it is corrected into a high-quality oriented anchor point through the regression branch, and the offset of the predicted anchor box regression target is predicted: Among them, (x, y, w, h, θ) represent the center coordinates, width, height, and angle parameters of the initial anchor point; (x g , y g , w g , h g , θ g ) represent the center coordinates, width, height, and angle parameters of the true bounding box; (Δx g , Δy g , Δw g , Δh g , Δθ g ) represent the offsets between the true bounding box and the initial anchor point. By regressing these offsets, the model can adjust the initial anchor point to a corrected predicted anchor box closer to the true bounding box; R(θ) represents a rotation transformation matrix used to convert the center point coordinates of the true bounding box into the coordinate system relative to the initial anchor point; k represents a scale factor used to adjust the angle value to ensure that the rotation angle is within a reasonable range; Use the corrected predicted anchor box as a guide to adjust the positions of the feature sampling points to achieve dynamic convolution alignment. On the basis of the original sampling points of the standard convolution, add an offset o deduced from the corrected predicted anchor box: Among them, represents the position of the predicted anchor box sampling point; (p0 + p n ) represents the position of the regular sampling point of the standard convolution; p0 and p n respectively represent the two-dimensional coordinates and relative offsets of the sampling point, and R represents the regular grid of the standard convolution [(p x , p y )]; The dynamic alignment convolution combines the offset o and the input feature x, so that the positions of the sampling points are adjusted according to the shape and direction of the predicted anchor box to match the geometric characteristics of the actual target: Among them, Y(p) represents the value of the output feature map at position p, and w(p n ) is the weight of the convolutional kernel at position p n , and x(·) represents the feature map of the input after feature fusion.

8. A sample optimization system, characterized in that, It includes a processing module, and the processing module executes the method described in any one of claims 1-3 to complete sample optimization.

9. A detection system, characterized in that, It includes a processing unit, and the processing unit includes a Dynamic Refinement Rotation Convolution Module (DRRCM) and an Anchor Refinement Feature Alignment Module (ARFAM). The processing unit executes the method according to one of claims 4-7 to obtain the remote sensing image target detection result.

Images