Real-time single-stage remote sensing image correction target detection method based on YOLOV8

Abstract

The invention discloses a real-time single-stage remote sensing image correction target detection method based on YOLOV8, and relates to the technical field of remote sensing image processing. According to the method, a deformable convolution dynamic prediction local geometric distortion parameter is embedded based on a YOLOv8 backbone network, an adaptive deformation field is generated, pixel-level real-time correction is realized, shallow details and high-level semantic features are fused through a bidirectional path aggregation network, and channel attention and a space gating mechanism are combined, so that the real-time correction of the image is realized. The small target detection capability is enhanced, background noise is suppressed, angle prediction is divided into discrete classification and continuous residual error regression tasks through a decoupling type rotation detection head, angle periodic errors are eliminated in combination with a direction sensitive loss function, and the rotation frame positioning precision is improved. And constructing a dynamic multi-task collaborative loss function, introducing gradient distribution consistency constraint to jointly optimize correction and detection tasks, and realizing feature semantic alignment and model self-enhancement through end-to-end closed-loop training. And the rotating target detection precision and the complex scene robustness are obviously improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and specifically to a real-time single-stage remote sensing image correction target detection method based on YOLOv8. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images are increasingly being used in a variety of fields, particularly in environmental monitoring, disaster assessment, agricultural monitoring, and urban planning. Automatic interpretation of remote sensing images not only enables relevant departments to achieve rapid, real-time monitoring of large geographic areas, but also provides critical data support for post-disaster rescue and resource allocation. Among these applications, target detection, as one of the core technologies for remote sensing image interpretation, can quickly and accurately identify and locate important targets in images. However, due to the high resolution and complex background of remote sensing images, target detection tasks face numerous challenges, such as detecting small targets, interference from complex backgrounds, and changes in target rotation angles. Therefore, improving the accuracy and efficiency of target detection in remote sensing images, especially for real-time detection in resource-constrained environments, has become a hot topic in current remote sensing research.

[0003] Traditional object detection methods have achieved some success in terrestrial computing environments, but they still face significant limitations in remote sensing imagery. First, objects in remote sensing images vary greatly in scale and rotation angle. Existing methods generally use horizontal bounding boxes with fixed scale and orientation for object localization. This approach cannot effectively handle rotated objects, resulting in degraded performance. Second, most of these methods rely on complex feature extraction and multi-stage processing, which is computationally intensive and slow to infer, making them difficult to meet the real-time processing requirements of resource-constrained environments such as orbit. Existing detection methods exhibit low accuracy for small objects and complex backgrounds and are significantly affected by bounding box errors, making them prone to missed and false detections. Furthermore, for the detection of rotated objects, most methods suffer from prediction errors in rotation angle and bounding box adjustments, significantly degrading performance for objects with large rotation angles or irregular shapes. Therefore, addressing these issues, particularly improving inference speed while maintaining detection accuracy, remains a major challenge in current object detection research in remote sensing images. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the present invention provides a real-time single-stage remote sensing image correction target detection method based on YOLOV8, which solves the problems of the above-mentioned background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a real-time single-stage remote sensing image correction target detection method based on YOLOv8, comprising the following steps: S1. Acquire remote sensing image data of the correction target, dynamically predict the local geometric distortion parameters of the correction target through a feature extraction network, generate an adaptive deformation field, and perform real-time correction on the geometric distortion caused by terrain undulation or sensor tilt in the remote sensing image; S2. Input the corrected image into the backbone network of YOLOv8 to extract a multi-scale feature map, fuse shallow detail features with high-level semantic features through a bidirectional path aggregation network, and output an enhanced multi-granularity feature map; S3. Based on the multi-granularity feature map, predict the target rotation bounding box parameters, including the center point coordinates, width and height dimensions, and discretized rotation angle classification results, through a decoupled rotation detection head, and optimize the detection accuracy through a direction-sensitive loss function; S4. According to the direction-sensitive loss value and the gradient distribution consistency constraint of the corrected image, construct a dynamic multi-task collaborative loss function, jointly optimize the network parameters of geometric correction and target detection, and realize end-to-end closed-loop iterative training.

[0006] Furthermore, the specific process of dynamically predicting the local geometric distortion parameters of the correction target through the feature extraction network and generating an adaptive deformation field is as follows: the backbone network based on YOLOv8 extracts multi-level features of the remote sensing image, and dynamically learns the geometric deformation pattern of the local area of ​​the correction target by embedding deformable convolution layers in the shallow features; the multi-level features are input into the deformation parameter prediction branch, and the pixel-level deformation field is generated through residual learning, where the deformation field contains the spatial offset of each pixel point; the original image is adaptively sampled through the deformation field, and the receptive field of the feature extraction network is dynamically adjusted to focus on the key structure of the distorted area of ​​the correction target.

[0007] Furthermore, the specific process of real-time correction of geometric distortion caused by terrain undulation or sensor tilt in remote sensing images is as follows: the generated deformation field is applied to the original remote sensing image, and the distorted area of ​​the correction target is resampled at the pixel level through the bilinear interpolation algorithm to eliminate the spatial misalignment caused by terrain or sensor tilt; feature pre-alignment is introduced in the corrected image, and the correction features are aligned with the standard spatial coordinate system through a lightweight convolutional network to reduce the positioning offset of the correction target; through an end-to-end training strategy, the network parameters of deformation field prediction and feature pre-alignment are jointly optimized to dynamically adapt to the geometric distortion characteristics under different terrain scenarios.

[0008] Furthermore, the shallow detail features and high-level semantic features are fused through a bidirectional path aggregation network to output an enhanced multi-granularity feature map. The specific process is as follows: in the top-down path, the high-level semantic features are upsampled and dynamically fused with the shallow high-resolution features through the channel attention mechanism to enhance the contextual information expression ability of the correction target; in the bottom-up path, the shallow detail features are enhanced by dilated convolution, and effective local features are screened through the spatial gating mechanism, and cross-scale interaction is carried out with the downsampled high-level features; through the interactive feature reorganization of the bidirectional path, a multi-granularity fused feature map of the correction target is generated.

[0009] Furthermore, based on multi-granularity feature maps, the specific process of predicting the target rotation bounding box parameters through a decoupled rotation detection head is as follows: the rotation angle prediction of the corrected target is decoupled into a discrete direction classification task and a continuous angle residual regression task, and the angle is discretized into multiple direction categories through a circular encoding strategy; a direction-sensitive attention mechanism is embedded in the detection process, and the feature weights are dynamically adjusted according to the target rotation angle to enhance the feature response of the rotation-sensitive area; the bounding box position and angle error are jointly optimized through the direction-sensitive intersection-over-union loss function, and the loss weights of the classification and regression tasks are dynamically balanced.

[0010] Furthermore, the specific process of optimizing detection accuracy through the direction-sensitive loss function is as follows: the direction-sensitive intersection-over-union loss is calculated based on the classification and regression results of the rotation angle of the correction target; the loss weight ratio of the correction target is dynamically adjusted through the gradient balancing mechanism to avoid detection bias; and the periodic constraint of the rotation angle is introduced to prevent confusion errors in angle prediction caused by periodic symmetry.

[0011] Furthermore, based on the consistency constraint of the direction-sensitive loss value and the gradient distribution of the corrected image, the specific process of constructing a dynamic multi-task collaborative loss function is as follows: the deformation field prediction error and the direction-sensitive intersection-over-union loss of the corrected target are weighted superimposed as the basic multi-task loss; the loss weight ratio of the correction task and the detection task is dynamically adjusted through the gradient balancing mechanism, and the gradient distribution consistency constraint of the corrected image and the standard undistorted image is introduced to force the correction features and the target detection features to be aligned in the semantic space.

[0012] Furthermore, the network parameters of geometric correction and target detection are jointly optimized to achieve end-to-end closed-loop iterative training. The specific process is as follows: during the forward propagation process, geometric correction, feature fusion, and rotation detection tasks are performed in sequence to generate the corrected image, multi-granularity feature map, and rotation box parameters; during the backpropagation process, the gradient is calculated through a dynamic multi-task collaborative loss function, and the network parameters of geometric correction and target detection are updated layer by layer based on the chain rule; the network parameters are optimized through iterative training, and the output image feature distribution is adaptively matched with the rotation-sensitive feature requirements of target detection, forming a self-enhanced detection closed loop.

[0013] The present invention has the following beneficial effects:

[0014] (1) A real-time single-stage remote sensing image correction target detection method based on YOLOv8: Based on the YOLOv8 feature extraction network, the local distortion parameters are dynamically predicted, and the adaptive deformation field is combined to achieve pixel-level geometric correction. This solves the problem of insufficient adaptability of traditional fixed parameter correction models in complex terrain scenes and significantly improves target positioning accuracy. The bidirectional path aggregation network fuses shallow details and high-level semantic features, effectively suppressing background noise while retaining small target details and enhancing small target detection capabilities. By utilizing the lightweight backbone network and bidirectional path aggregation architecture of YOLOv8, the number of model parameters is significantly reduced, the inference speed is improved, and the real-time processing requirements of embedded devices are met.

[0015] (2) A real-time single-stage remote sensing image rectification target detection method based on YOLOV8 is proposed. The decoupled rotation detection head divides angle prediction into discrete classification and continuous residual regression tasks. Combined with the direction-sensitive loss function, it effectively solves the angle periodic error problem in traditional rotation frame detection and improves the angle prediction accuracy. The dynamic multi-task collaborative loss function is used to jointly optimize the geometric correction and target detection tasks, and the gradient distribution consistency constraint is used to achieve feature semantic alignment, which significantly enhances the detection robustness of the model in complex scenarios. The end-to-end iterative training strategy enables the correction and detection network parameters to be adaptively matched without manual intervention in parameter adjustment, significantly improving the training efficiency and model generalization ability, and adapting to remote sensing data of different terrains and sensor types.

[0016] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a flow chart of the real-time single-stage remote sensing image correction target detection method based on YOLOV8 in the present invention. DETAILED DESCRIPTION

[0018] The embodiment of the present application solves the problem of geometric distortion of remote sensing images caused by factors such as terrain undulation or sensor tilt through a real-time single-stage remote sensing image correction target detection method based on YOLOV8, while improving the accuracy and real-time performance of rotating target detection. Traditional remote sensing image target detection methods often lead to low target detection accuracy due to ignoring the influence of image geometric distortion and target rotation angle, especially in complex terrain and under different viewing angles. This embodiment introduces an adaptive deformation field for real-time geometric correction, combined with multi-scale feature fusion and a decoupled rotation detection head, which not only improves the detection accuracy of rotating targets, but also effectively improves the reasoning speed of the system, making this method very suitable for real-time applications in embedded devices and mobile platforms with limited resources.

[0019] The overall idea of ​​the solution in the embodiments of this application is as follows:

[0020] Acquire remote sensing image data of the correction target, dynamically predict the local geometric distortion parameters of the correction target through the feature extraction network, generate an adaptive deformation field, and perform real-time correction on the geometric distortion caused by terrain undulation or sensor tilt in the remote sensing image.

[0021] The rectified image is input into the backbone network of YOLOv8 to extract multi-scale feature maps, and the shallow detail features and high-level semantic features are fused through the bidirectional path aggregation network to output enhanced multi-granularity feature maps.

[0022] Based on multi-granularity feature maps, the decoupled rotation detection head is used to predict the target rotation bounding box parameters, including the center point coordinates, width and height dimensions, and discretized rotation angle classification results, and the detection accuracy is optimized through a direction-sensitive loss function.

[0023] Based on the consistency constraint of the direction-sensitive loss value and the gradient distribution of the corrected image, a dynamic multi-task collaborative loss function is constructed to jointly optimize the network parameters of geometric correction and target detection to achieve end-to-end closed-loop iterative training.

[0024] See also Figure 1The embodiment of the present invention provides a technical solution: a real-time single-stage remote sensing image correction target detection method based on YOLOv8, comprising the following steps: S1. Acquire remote sensing image data of the correction target, dynamically predict the local geometric distortion parameters of the correction target through a feature extraction network, generate an adaptive deformation field, and perform real-time correction on the geometric distortion caused by terrain undulation or sensor tilt in the remote sensing image; S2. Input the corrected image into the backbone network of YOLOv8 to extract a multi-scale feature map, fuse shallow detail features with high-level semantic features through a bidirectional path aggregation network, and output an enhanced multi-granularity feature map; S3. Based on the multi-granularity feature map, predict the target rotation bounding box parameters, including the center point coordinates, width and height dimensions, and discretized rotation angle classification results, through a decoupled rotation detection head, and optimize the detection accuracy through a direction-sensitive loss function; S4. According to the direction-sensitive loss value and the gradient distribution consistency constraint of the corrected image, construct a dynamic multi-task collaborative loss function, jointly optimize the network parameters of geometric correction and target detection, and realize end-to-end closed-loop iterative training.

[0025] In this implementation, step 1: Acquire remote sensing image data containing the target. These images may exhibit geometric distortion due to terrain variations (such as mountainous, oceanic, and urban terrain) or sensor tilt (such as the angle of satellite or drone photography). To effectively detect targets, these geometric distortions must first be corrected. Feature extraction network: This involves extracting meaningful features (such as edges, textures, and shapes) from the input image using a deep learning model. These features can help identify targets in the image. Convolutional neural networks (CNNs) are typically used to extract these features. Local geometric distortion parameters: These refer to the degree of image distortion caused by factors such as terrain or device tilt. These distortions affect the precise location of the target and therefore require correction. Adaptive deformation field: Based on the predicted geometric distortion parameters, the image deformation is dynamically adjusted to restore the target in the image to its normal state. Step 2: The corrected image is input into the YOLOv8 backbone network for multi-scale feature extraction. The backbone network is responsible for extracting features at different levels from the image, including detailed information (such as edges and corners) and high-level semantic information (such as object category). These features will then be passed to the downstream network modules for target detection. YOLOv8 backbone network: The core structure of the YOLOv8 model, used to extract the basic features of the image. YOLOv8 uses the efficient CSPDarknet as the backbone network, which reduces the amount of computation and improves detection efficiency by separating and merging feature maps. Multi-scale feature maps: In target detection, the sizes of objects may be very different. Through multi-scale feature maps, targets of different sizes can be processed simultaneously to improve detection accuracy. Bidirectional Path Aggregation Network (PAFPN): This is a feature fusion mechanism that fuses feature maps of different scales through top-down (from high-level to low-level) and bottom-up paths (from low-level to high-level) to enhance the model's perception of details and semantics, thereby improving the performance of target detection. Step 3 In this step, the multi-granularity feature map generated by YOLOv8 is used to predict the oriented bounding box (OBB) of the target using a decoupled rotation detection head (Head). Object detection not only needs to predict the position (center point coordinates) and size (width and height) of the object, but also needs to consider the rotation angle of the object. In this process, a direction-sensitive loss function is used to optimize the prediction accuracy of the rotation angle. Decoupled rotation detection head: refers to a network structure specifically used for rotation object detection, through which the rotation angle, position and size of the object can be accurately predicted. Unlike the traditional horizontal bounding box (HBB), the rotation bounding box can handle tilted or rotated objects, and is particularly suitable for complex scenes. Rotation bounding box (OBB): is a bounding box that can represent the rotation angle of the object. Unlike the traditional horizontal bounding box (HBB), OBB allows the prediction of arbitrary rotation angles of the object, thereby more accurately adapting to the detection of rotated objects in the image.Direction-Sensitive Loss: A loss function that specifically considers the object's rotation angle when calculating the object detection error. This allows the model to better handle rotated objects and improves localization accuracy. Step 4: In this step, a dynamic multi-task collaborative loss function is constructed to combine the optimization processes of geometric correction and object detection. This means optimizing not only the accuracy of object detection but also the accuracy of image geometric correction. This allows for joint optimization of the two tasks, ensuring that the final model can simultaneously improve object detection performance while performing geometric correction. Dynamic Multi-Task Collaborative Loss: A loss function that simultaneously handles multiple tasks (such as geometric correction and object detection). By constraining the relationship between these two tasks, it ensures that the effects of image geometric correction are not neglected during object detection, and vice versa. This avoids performance bottlenecks caused by optimizing a single task. Gradient Consistency Constraint: A constraint that ensures that changes to edges or specific regions of an image remain consistent during the correction process. This constraint optimizes the accuracy of image correction, allowing the object detection network to be trained on more accurate input images.

[0026] Specifically, the specific process of dynamically predicting the local geometric distortion parameters of the correction target through the feature extraction network and generating an adaptive deformation field is as follows: the YOLOv8-based backbone network extracts multi-level features of the remote sensing image, and dynamically learns the geometric deformation pattern of the local area of ​​the correction target by embedding deformable convolution layers in the shallow features; the multi-level features are input into the deformation parameter prediction branch, and the pixel-level deformation field is generated through residual learning, where the deformation field contains the spatial offset of each pixel point; the original image is adaptively sampled through the deformation field, and the receptive field of the feature extraction network is dynamically adjusted to focus on the key structure of the distorted area of ​​the correction target.

[0027] In this implementation, multi-level features of remote sensing images are extracted based on the YOLOv8 backbone network. YOLOv8 (YouOnly Look Once version 8) is an efficient object detection network that utilizes a deep convolutional neural network (CNN) architecture and is capable of extracting multi-level features from images in real time. Its backbone network is responsible for extracting features at different levels from the input remote sensing image. Typically, low-level features represent edges, textures, and detail information in an image, while high-level features focus on semantic information in the image, such as object categories and large-scale structures. The YOLOv8 backbone network first extracts these multi-level features (i.e., shallow, mid-level, and deep-level features) from the remote sensing image. The features of each layer contain information at different scales and levels of abstraction, providing rich contextual information for subsequent image correction. Deformable convolution layers are embedded in the shallow-level features to dynamically learn geometric deformation patterns: Deformable convolution is a technique that extends the standard convolution operation, allowing the convolution kernel to flexibly adjust its sampling position based on the local geometric deformation of the input image. This means that the fixed grid sampling positions in traditional convolution operations (i.e., the fixed weight positions for each pixel within the convolution kernel) can be dynamically adjusted based on local geometric changes in the image. Deformable convolutions are applied to the shallow features of the YOLOv8 network. Shallow features typically contain image details such as edges and texture, which are often crucial for correcting geometric distortion. By embedding deformable convolution layers within these shallow features, the network can dynamically learn and capture geometric deformation patterns in remote sensing images caused by factors such as terrain undulation or sensor tilt. In other words, the network can automatically adjust the position of the convolution kernel to better adapt to the characteristics of locally distorted areas, thereby more effectively predicting and correcting geometric distortion. Multi-level features are input into the deformation parameter prediction branch, which generates a pixel-level deformation field through residual learning. This branch's task is to generate a deformation field, which is the spatial offset (in the X and Y directions) of each pixel in the image. This offset represents the position to which each pixel should be moved after geometric correction. By introducing residual learning, the deformation field prediction process becomes more efficient and stable. Residual learning trains the network to learn the difference between the predicted image and the rectified image, rather than directly learning the rectified image itself. This allows the network to more easily capture subtle changes in the image and improve the accuracy of the deformation field. The original image is adaptively sampled using the deformation field, and the resulting deformation field (including the spatial offset of each pixel) is used to adaptively sample the original image. Adaptive sampling involves adjusting the pixels in the image based on the offsets predicted in the deformation field. Each pixel is "stretched" or "compressed" according to its corresponding offset to eliminate geometric distortion caused by terrain undulations or sensor tilt.In this way, deformation in each local area of ​​the image can be corrected, restoring the image's geometric structure. Adaptive sampling not only eliminates global geometric distortion but also precisely corrects local distorted areas, particularly those with significant terrain changes or sensor tilt. The receptive field of the feature extraction network is dynamically adjusted to focus on the distorted areas of the correction target. The receptive field refers to the size of the image area that the network can "see" when processing a specific pixel. By dynamically adjusting the receptive field, the network can more flexibly focus on information at different scales. The image corrected by the deformation field is then input into the subsequent feature extraction network. At this point, the network can adaptively adjust its receptive field based on the deformation field's cues, focusing on areas with significant distortion. This allows the feature extraction network to focus on key structural components of the correction target, such as buildings, roads, or other areas of interest in remote sensing images, further improving object detection accuracy.

[0028] Specifically, the specific process of real-time correction of geometric distortion caused by terrain undulation or sensor tilt in remote sensing images is as follows: the generated deformation field is applied to the original remote sensing image, and the distorted area of ​​the correction target is resampled at the pixel level through the bilinear interpolation algorithm to eliminate the spatial misalignment caused by terrain or sensor tilt; feature pre-alignment is introduced in the corrected image, and the correction features are aligned with the standard spatial coordinate system through a lightweight convolutional network to reduce the positioning offset of the correction target; through an end-to-end training strategy, the network parameters of deformation field prediction and feature pre-alignment are jointly optimized to dynamically adapt to the geometric distortion characteristics under different terrain scenarios.

[0029] In this embodiment, the remote sensing image is geometrically corrected by generating a deformation field. The deformation field is used to describe the spatial offset of each pixel in the image. Through these offsets, the distortion in the image caused by terrain undulation or sensor tilt can be corrected. Specifically, the deformation field calculates the displacement of each pixel in the X and Y directions. Based on these displacements, we can move each pixel in the original image to a new position, thereby eliminating the geometric distortion. In order to eliminate the distortion in the image, the pixels in the image are resampled using a bilinear interpolation algorithm. The bilinear interpolation method calculates the new pixel value through the values ​​of four adjacent pixels, and uses the weights between adjacent pixels to interpolate to ensure that the image remains smooth and continuous after distortion repair. In this way, the spatial misalignment caused by terrain or sensor tilt can be eliminated. Even if the image is corrected by the deformation field, the corrected image may still have some slight positioning offsets, which may affect subsequent target detection tasks. In order to further reduce this offset, feature pre-alignment technology is used. A lightweight convolutional neural network (CNN) aligns the corrected image features and maps them to a standard spatial coordinate system, thereby reducing the localization error of objects in the image. This process ensures that the image features more accurately reflect the true location of the object. To achieve joint optimization of geometric distortion and object detection, an end-to-end training strategy is adopted. This strategy means that all model components (such as deformation field prediction, feature alignment, and object detection) share parameters during training and are optimized using a unified objective function. Specifically, the network optimizes not only the deformation field prediction error but also the feature alignment. Through this joint training, the network can dynamically adjust network parameters to achieve optimal distortion and detection performance when handling different terrains and scenes. In practical applications, remote sensing images from different terrains experience different geometric distortion characteristics. Therefore, the network needs to be able to dynamically adapt to these different terrain characteristics (such as mountainous areas, plains, and urban areas). Through end-to-end training, the network learns how to adaptively adjust to images from different terrains, thereby improving the robustness of geometric distortion and the accuracy of object detection.

[0030] Specifically, the bidirectional path aggregation network is used to fuse shallow detail features and high-level semantic features to output an enhanced multi-granularity feature map. The specific process is as follows: in the top-down path, the high-level semantic features are upsampled and dynamically fused with the shallow high-resolution features through the channel attention mechanism to enhance the contextual information expression ability of the correction target; in the bottom-up path, the shallow detail features are enhanced by dilated convolution, and the effective local features are screened through the spatial gating mechanism, and cross-scale interaction is carried out with the downsampled high-level features; through the interactive feature reorganization of the bidirectional path, a multi-granularity fused feature map of the correction target is generated.

[0031] In this implementation, the top-down path dynamically integrates upsampling and channel-wise attention. In the top-down path of the bidirectional path aggregation network, we first upsample high-level semantic features. Upsampling involves scaling low-resolution feature maps using an interpolation method (such as bilinear interpolation) to restore the spatial dimensions of the original image. At this stage, high-level semantic features typically contain less detailed information but are crucial for understanding the image's global structure and semantics. The channel-wise attention mechanism dynamically adjusts the weights of different feature channels to enable the network to focus on more important features. During this process, the network assigns different weights to each channel based on the magnitude of the input feature's response. This allows the network to highlight features that have a greater impact on the correction target (e.g., distorted regions) while suppressing less relevant features. The goal of this step is to enhance the image's ability to express contextual information by fusing high-level semantic features with shallow, high-resolution features, enabling the network to more accurately understand and correct distorted regions in remote sensing imagery. Bottom-up Path: Dilated Convolution and Spatial Gating Mechanism. In the bottom-up path of the bidirectional path aggregation network, shallow detail features are enhanced through dilated convolution. Dilated convolution introduces "gaps" between the weights of the convolution kernel, increasing the receptive field without increasing computational effort, thus capturing a wider range of contextual information. Dilated convolution is particularly effective for processing detailed information because it preserves local details while expanding the receptive range. Next, a spatial gating mechanism is applied to filter shallow detail features. Spatial gating introduces conditional constraints (typically based on features at specific spatial locations) to selectively activate or suppress certain features spatially. This mechanism effectively selects the most useful local features in different regions, thereby enhancing the network's focus on important information. The goal of this step is to further enrich the semantic information of the image, particularly details such as distorted areas or key information, by enhancing shallow detail features and interacting them with downsampled high-level semantic features across scales. After completing the operations of the top-down and bottom-up paths, the network interactively recombines the features extracted from the two paths. The core idea of ​​this step is to combine high-level semantic features with shallow, detailed features through bidirectional feature interaction to generate feature maps with multi-granular information. Interactive feature recombination involves fusing features from different paths, allowing each feature to represent image information from different scales and perspectives, thereby generating richer and more comprehensive feature maps. These feature maps not only contain global semantic information but also preserve detailed and local information, helping to further improve the accuracy of image correction.

[0032] Specifically, based on multi-granularity feature maps, the specific process of predicting the target rotation bounding box parameters through a decoupled rotation detection head is as follows: the rotation angle prediction of the corrected target is decoupled into a discrete direction classification task and a continuous angle residual regression task, and the angle is discretized into multiple direction categories through a circular encoding strategy; a direction-sensitive attention mechanism is embedded in the detection process, and the feature weights are dynamically adjusted according to the target rotation angle to enhance the feature response of the rotation-sensitive area; the bounding box position and angle error are jointly optimized through the direction-sensitive intersection-over-union loss function, and the loss weights of the classification and regression tasks are dynamically balanced.

[0033] In this embodiment, the decoupled task design of rotation angle prediction: the core of this process is to predict the rotation bounding box parameters of the target through a decoupled rotation detection head. Specifically, the rotation angle prediction of the target is decomposed into two tasks: discrete direction classification task: discretize the rotation angle into multiple direction categories, making angle prediction simpler. The discretized angle is mapped to a finite category, each category corresponds to a direction. Through this method, rotation angle prediction is no longer a continuous regression problem, but a multi-classification problem. Continuous angle error regression task: predict the deviation between the discretized angle category and the actual angle. Through the regression task, the model can further accurately predict the actual value of the angle and correct the error caused by discretization. In order to discretize the angle into multiple direction categories, a ring coding strategy is used. Ring coding is a technique that maps angles (usually values ​​within 360 degrees) to a finite number of categories. Due to the periodicity of angles (for example, 0 degrees equals 360 degrees), ring coding takes this into account, so that the model is not affected by the periodicity of angles when learning. Assuming the angle range is θ∈[0,2π), the angle can be discretized into N direction categories through ring coding. Each category c i Corresponding to a discrete angle range, and represented by encoding. The form of ring encoding can be achieved by mapping the angle θ to an N-dimensional unit vector, each vector corresponding to a direction category: v i =[cos(θ i ),sin(θ i )]; where θ i is the discretized angle, i∈{1,2,…,N}, representing the corresponding direction category. Direction-sensitive attention mechanism: In order to enhance the feature response of the rotation-sensitive area, the network introduces a direction-sensitive attention mechanism. The goal of this mechanism is to dynamically adjust the weights of different regions in the feature map according to the rotation angle of the target, so that the features of the rotation-sensitive area receive stronger attention. Given the rotation angle θ of the target target , by calculating the direction-sensitive attention weight to adjust the response in the feature map. For a certain feature map position p, the direction-sensitive attention mechanism calculates the weight A(p) of the position: A(p) = f(θtarget ,p); where f(·) represents a mapping function based on the target rotation angle and spatial position p, which can be a lightweight convolutional neural network that outputs a weight value related to the target angle. The network can adaptively enhance the influence of the rotation area. The target bounding box includes not only the position (center coordinates) and size (width and height), but also the rotation angle. In order to simultaneously optimize the rotation angle and bounding box position of the target, the direction-sensitive intersection-over-union loss function (DirectionalIoULoss) is adopted. The intersection-over-union (IoU) is used to measure the degree of overlap between the predicted box and the true box, and the direction-sensitive intersection-over-union loss function takes into account the characteristics of the rotated target and adjusts the standard IoU calculation so that the rotation angle can also be included in the optimization. Assume that the rotation angle of the predicted box is The rotation angle of the real frame is θ gt , then the direction-sensitive intersection-over-union loss function can be expressed as: in: and B gt are the coordinates and sizes of the predicted and true bounding boxes, respectively. and θ gt are the rotation angles of the predicted and ground-truth bounding boxes, respectively. λ0 is a hyperparameter that controls the impact of rotation angle errors on the loss. A larger λ0 increases the penalty for rotation angle errors. Object detection tasks typically involve simultaneous classification (i.e., predicting the object category) and regression (i.e., predicting the bounding box position and rotation angle). To ensure a reasonable balance between the losses of these two tasks, a dynamic loss weight can be introduced that is dynamically adjusted based on the difficulty of the task.

[0034] Specifically, the specific process of optimizing detection accuracy through the direction-sensitive loss function is as follows: according to the classification and regression results of the rotation angle of the correction target, the direction-sensitive intersection-over-union loss is calculated; the loss weight ratio of the correction target is dynamically adjusted through the gradient balancing mechanism to avoid detection bias; and the periodic constraint of the rotation angle is introduced to prevent the confusion error caused by the periodic symmetry of the angle prediction.

[0035] In this implementation, the direction-sensitive intersection-over-union (IoU) loss function is designed to optimize bounding box detection accuracy for rotated objects. Specifically, when the object is rotated, it considers not only the bounding box's position and size but also the object's rotation angle. The core idea of ​​this loss function is to incorporate the rotation angle factor into the standard intersection-over-union (IoU) loss. IoU is a metric that measures the overlap between the predicted and ground-truth bounding boxes, expressed as the ratio of the area of ​​the intersection of the predicted and ground-truth bounding boxes to the area of ​​their union. In the case of rotated objects, IoU should also consider the impact of the rotation angle, namely the degree of overlap between the rotated and ground-truth bounding boxes. During deep learning training, the losses for different tasks may have different numerical ranges, leading to uneven gradient updates and, in turn, impacting the training process. To mitigate the differential effects of rotation angle error and position error on network training, a gradient balancing mechanism is introduced to dynamically adjust the weighting of the losses for each task. The goal of gradient balancing is to dynamically adjust the loss weights for different tasks (such as rotation angle regression and bounding box repositioning) based on the magnitude and variation of the gradients. In this way, the network can adaptively balance the optimization of these two tasks during training. The gradient balancing mechanism is updated as follows: Where: α t It is the loss weight ratio at the current moment, which represents the gradient weight ratio of the regression task (position and rotation angle) and the classification task. and are the gradients of the loss functions for regression and classification tasks, respectively. Due to the periodic nature of angles (e.g., 0° and 360° are equivalent), confusion errors caused by periodic symmetry may occur in angle regression tasks. For example, the network may predict an angle as θ = 359°, while the true angle is θ gt = 1°, which is obviously wrong, but they may be considered reasonable in ordinary regression tasks. To solve this problem, a periodic constraint on the rotation angle is introduced. This constraint avoids confusion caused by periodic symmetry by adjusting the loss function. Specifically, for the predicted rotation angle θ and the true angle θ gt , we calculate the minimum angle difference between them: The minimum angle difference is then used as the angle error to calculate the loss. In this way, the model can adapt to the periodicity of the angle and avoid predicting the wrong rotation angle.

[0036] Specifically, based on the consistency constraint of the direction-sensitive loss value and the gradient distribution of the corrected image, the specific process of constructing the dynamic multi-task collaborative loss function is as follows: the deformation field prediction error and the direction-sensitive intersection-over-union loss of the corrected target are weighted superimposed as the basic multi-task loss; the loss weight ratio of the correction task and the detection task is dynamically adjusted through the gradient balancing mechanism, and the gradient distribution consistency constraint of the corrected image and the standard undistorted image is introduced to force the correction features and the target detection features to be aligned in the semantic space.

[0037] In this implementation, we introduce a gradient distribution consistency constraint to ensure that the rectified image is consistent with the standard undistorted image in the gradient space. The goal of this constraint is to ensure that the rectified image aligns with the semantic features of the object detection task in the feature space by comparing the gradient distribution differences between the rectified image and the standard undistorted image. The gradient distribution consistency constraint is calculated as follows: Where: L grad It is the gradient distribution consistency loss, which measures the difference between the gradient of the rectified image and the gradient distribution of the standard undistorted image. and They are the rectified image and the standard undistorted image at pixel x i The gradient at x i Is a pixel at a certain position in the image. The final dynamic multi-task collaborative loss function combines the basic multi-task loss, gradient balancing mechanism and gradient distribution consistency constraint. Our final dynamic multi-task collaborative loss function is: L total =α t (α1L deformation +α2L IoU )+λ grad L grad ; Among them: α t is the loss weight ratio dynamically adjusted by the gradient balancing mechanism. α1 and α2 are the weight parameters of the deformation field prediction error and direction-sensitive intersection-over-union loss, respectively. grad is the gradient distribution consistency loss, which ensures that the rectified image is consistent with the standard undistorted image in the gradient space. grad L is a hyperparameter of the gradient distribution consistency constraint, which is used to control the degree of penalty for the gradient distribution consistency loss. deformation L is the deformation field prediction error, which is used to measure the restoration quality of image geometric distortion. IoU It is a direction-sensitive intersection-over-union loss used to optimize target detection tasks.

[0038] Specifically, the network parameters of geometric correction and target detection are jointly optimized to achieve end-to-end closed-loop iterative training. The specific process is as follows: during the forward propagation process, geometric correction, feature fusion, and rotation detection tasks are performed in sequence to generate the corrected image, multi-granularity feature map, and rotation box parameters; during the backpropagation process, the gradient is calculated through a dynamic multi-task collaborative loss function, and the network parameters of geometric correction and target detection are updated layer by layer based on the chain rule; the network parameters are optimized through iterative training, and the output image feature distribution is adaptively matched with the rotation-sensitive feature requirements of target detection, forming a self-enhanced detection closed loop.

[0039] In this implementation, when jointly optimizing the network parameters for geometric correction and object detection, our goal is to enable the network to simultaneously optimize both image geometric correction and object detection through end-to-end training. This process not only improves the accuracy of geometric correction but also enhances the accuracy of object detection, ultimately forming a self-enhancing detection closed loop. During the forward propagation phase, the entire network performs a series of operations from image input to final output. The forward propagation process consists of three main steps: First, the input remote sensing image passes through the geometric correction module, which uses the previously mentioned deformation field prediction and bilinear interpolation algorithm to remove distortion caused by terrain undulation or sensor tilt. By predicting the deformation field and performing pixel-level resampling, the network transforms the original image into a corrected image, reducing spatial misalignment. Next, a bidirectional path aggregation network fuses multi-granular features to generate a multi-level feature map. This process enriches the semantic information and detail representation of the image by upsampling, fusing, and enhancing shallow-level detail features with high-level semantic features using dilated convolutions. Finally, the corrected image and multi-granular feature map are input to the rotation detection module. This module uses a decoupled rotation detection head to predict the parameters of the target's rotation bounding box, including the target's rotation angle and bounding box position. Using a direction-sensitive intersection-over-union loss function, the rotation angle and bounding box regression tasks are optimized under the guidance of a multi-task collaborative loss function.

[0040] During the backpropagation phase, the network parameters are updated based on the gradient of the loss function. This process consists of the following steps: The network calculates the loss for each task using a dynamic multi-task collaborative loss function. This loss function comprehensively considers the deformation field prediction error, the orientation-sensitive intersection-over-union loss, and the gradient distribution consistency constraint. The calculated loss gradients are used to guide the network parameter updates during backpropagation. Based on the calculated losses, the network parameters are updated layer by layer using the chain rule during backpropagation. The gradients of each layer are propagated to the previous layer using the chain rule, adjusting the corresponding weights and biases. This allows the network to optimize the network parameters of each module based on the feedback signals from the geometric correction and object detection tasks. The gradients calculated through backpropagation are used to update the parameters of each module of the network (geometric correction module, feature fusion module, and rotation detection module). Because the loss function integrates the losses of multiple tasks, the parameter updates of each module take into account the interrelationships between the multiple tasks. Object detection network parameters: These include parameters of the rotated bounding box regression network, orientation-sensitive attention mechanism, and others. During iterative training, through multiple forward and backward passes, the network gradually optimizes its parameters, improving the performance of both geometric correction and object detection. As training progresses, the loss function gradually decreases, allowing the network to adaptively match the rotation-sensitive features required for object detection, ultimately forming a self-enhancing detection loop. During training, the network not only optimizes the geometric correction task but also automatically adjusts the distribution of image features to adaptively match the requirements of the object detection task by dynamically adjusting weights, gradient balancing, and orientation-sensitive loss. As geometric correction performance improves, object detection performance also improves, and vice versa, forming a positive feedback loop. This self-enhancing loop enables the network to dynamically adapt to different terrain or distortion scenarios, gradually improving the overall performance of both tasks.

[0041] In summary, this application has at least the following effects:

[0042] A real-time, single-stage object detection method for remote sensing image rectification based on YOLO V8 effectively eliminates geometric distortion caused by terrain undulation or sensor tilt through deformation field prediction and bilinear interpolation resampling, significantly improving the spatial alignment accuracy of remote sensing images. Multi-granular feature fusion and rotation-sensitive object detection accurately predict the position and angle of rotated bounding boxes, improving object detection accuracy, especially in complex terrain or distorted scenes. By introducing a lightweight convolutional network and a bidirectional path aggregation network, deep feature fusion and dynamic weight adjustment optimize the feature synergy between the rectification and object detection tasks, enhancing the representation of contextual information and local features. By dynamically adjusting the weight ratios of multi-task losses and introducing gradient distribution consistency constraints, combined with a direction-sensitive rotation detection mechanism, the method achieves joint optimization of geometric rectification and object detection, ensuring balance and adaptive adjustment between the different tasks. Through end-to-end closed-loop training, the geometric rectification and object detection tasks mutually reinforce each other, forming an adaptive matching and self-reinforcement closed loop, which can continuously improve the overall performance of image rectification and object detection under various terrain and distortion conditions. Combining multi-level features and direction-sensitive loss functions enables the model to maintain high robustness and accuracy in the face of factors such as different terrains, climate changes, and sensor errors.

[0043] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0044] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0045] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0046] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0047] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0048] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A real-time single-stage remote sensing image rectification target detection method based on YOLOV8, characterized by: The following steps are involved: S1. Acquire remote sensing image data of the correction target, dynamically predict the local geometric distortion parameters of the correction target through a feature extraction network, generate an adaptive deformation field, and perform real-time correction of geometric distortion caused by terrain undulation or sensor tilt in the remote sensing image. S2. Input the rectified image into the YOLOv8 backbone network to extract multi-scale feature maps. The bidirectional path aggregation network fuses shallow detail features with high-level semantic features to output enhanced multi-granularity feature maps. S3. Based on multi-granular feature maps, a decoupled rotation detection head is used to predict the target's rotation bounding box parameters, including center point coordinates, width and height dimensions, and discretized rotation angle classification results. Detection accuracy is optimized using a direction-sensitive loss function. S4. Based on the consistency constraint between the direction-sensitive loss value and the gradient distribution of the corrected image, a dynamic multi-task collaborative loss function is constructed to jointly optimize the network parameters of geometric correction and target detection to achieve end-to-end closed-loop iterative training.

2. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 1 is characterized in that: The specific process of dynamically predicting and correcting the local geometric distortion parameters of the target through the feature extraction network and generating the adaptive deformation field is as follows: The backbone network based on YOLOv8 extracts multi-level features of remote sensing images and dynamically learns to correct the geometric deformation patterns of local target areas by embedding deformable convolutional layers in shallow features. Multi-level features are input into the deformation parameter prediction branch, and a pixel-level deformation field is generated through residual learning, where the deformation field contains the spatial offset of each pixel; The original image is adaptively sampled through the deformation field, and the receptive field of the feature extraction network is dynamically adjusted to focus on the key structures in the distorted area of ​​the correction target.

3. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 2 is characterized in that: The specific process of real-time correction of geometric distortion in remote sensing images caused by terrain undulation or sensor tilt is as follows: The generated deformation field is applied to the original remote sensing image, and the distorted area of ​​the correction target is resampled at the pixel level using a bilinear interpolation algorithm to eliminate spatial misalignment caused by terrain or sensor tilt. Feature pre-alignment is introduced into the rectified image, and the rectified features are aligned with the standard space coordinate system through a lightweight convolutional network to reduce the positioning offset of the rectified target; Through an end-to-end training strategy, the network parameters of deformation field prediction and feature pre-alignment are jointly optimized to dynamically adapt to the geometric distortion characteristics under different terrain scenarios.

4. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 3 is characterized by: The specific process of fusing shallow detail features and high-level semantic features through a bidirectional path aggregation network and outputting an enhanced multi-granularity feature map is as follows: In the top-down path, high-level semantic features are upsampled and dynamically fused with shallow high-resolution features through a channel attention mechanism to enhance the ability to express contextual information of the correction target. In the bottom-up path, shallow detail features are enhanced by dilated convolution, effective local features are filtered through a spatial gating mechanism, and cross-scale interaction is achieved with downsampled high-level features. Through interactive feature recombination of bidirectional paths, a multi-granularity fused feature map of the correction target is generated.

5. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 4 is characterized in that: The specific process of predicting the target rotation bounding box parameters using the decoupled rotation detection head based on the multi-granularity feature map is as follows: The rotation angle prediction of the correction target is decoupled into a discrete direction classification task and a continuous angle residual regression task, and the angle is discretized into multiple direction categories through a circular encoding strategy; Embedding a direction-sensitive attention mechanism in the detection process to dynamically adjust feature weights according to the target rotation angle and enhance the feature response of rotation-sensitive areas; The bounding box position and angle error are jointly optimized through the direction-sensitive intersection-over-union loss function, dynamically balancing the loss weights of classification and regression tasks.

6. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 5, characterized in that: The specific process of optimizing detection accuracy through direction-sensitive loss function is as follows: Calculate the direction-sensitive intersection-over-union loss based on the classification and regression results of the corrected target’s rotation angle; Dynamically adjust the loss weight ratio of the correction target through the gradient balancing mechanism to avoid detection bias; The periodic constraint of the rotation angle is introduced to prevent the aliasing error caused by periodic symmetry in the angle prediction.

7. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 6, characterized in that: According to the consistency constraint of the direction-sensitive loss value and the gradient distribution of the rectified image, the specific process of constructing the dynamic multi-task collaborative loss function is as follows: The deformation field prediction error and the direction-sensitive intersection-and-intersection loss of the correction target are weighted and superimposed as the basic multi-task loss. The loss weight ratio of the correction task and the detection task is dynamically adjusted through the gradient balancing mechanism, and the gradient distribution consistency constraint between the corrected image and the standard undistorted image is introduced to force the correction features and the target detection features to be aligned in the semantic space.

8. The real-time single-stage remote sensing image rectification target detection method based on YOLOv8 according to claim 7, characterized in that: The specific process of jointly optimizing the network parameters for geometric correction and object detection to achieve end-to-end closed-loop iterative training is as follows: During the forward propagation process, geometric correction, feature fusion, and rotation detection tasks are performed in sequence to generate the corrected image, multi-granularity feature map, and rotation box parameters; During backpropagation, the gradient is calculated through a dynamic multi-task collaborative loss function, and the network parameters for geometric correction and target detection are updated layer by layer based on the chain rule. Through iterative training to optimize network parameters, the output image feature distribution is adaptively matched with the rotation-sensitive feature requirements of target detection, forming a self-enhanced detection closed loop.

Images