Intelligent Detection Method for Aberration of Space Optical Remote Sensing Payload

Abstract

The present invention relates to the technical field of remote sensing images, and particularly to an intelligent detection method for aberrations of space optical remote sensing payloads. This method introduces the MDHA mechanism into the YOLOv8 network to obtain the YOLOv8_MA network, and uses the trained YOLOv8_MA model to perform aberration category classification detection on remote sensing images, realizing the automatic detection of star point targets in optical remote sensing images, and can efficiently deduce the aberration situation of the imaging system, thereby providing key information support for the on-orbit aberration adjustment of optical remote sensing payloads.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing images, and particularly relates to an intelligent aberration detection method for space optical remote sensing payloads. Background Technique

[0002] During the development of optical remote sensing payloads, their performance is affected by various factors, including ground gravity, vibration and shock during launch, and changes in the space gravity field and temperature after entering orbit. As the aperture and focal length of the optical system increase, the deformation of the optical system caused by factors such as gravity, vibration, shock, and temperature may lead to large wavefront aberrations, thereby affecting the imaging performance of the remote sensor. To solve this problem, advanced space remote sensors have adopted active optical correction technology to improve imaging quality. The aberration adjustment of optical remote sensing payloads mainly uses experience-based manual adjustment, which requires experienced operators for adjustment. However, with the development of technology, image detection technology based on deep learning has also been gradually applied to the aberration adjustment of optical remote sensing payloads, which can more accurately capture and correct complex aberration phenomena.

[0003] Before performing active optical correction, it is necessary to analyze the causes of aberrations in the imaging pictures according to the types of aberrations, so as to determine the correction positions in the imaging system. Specifically, astigmatism is mainly caused by the difference between the state of the optical system and the ideal optical system, resulting in inconsistent focusing of light rays on different planes; spherical aberration is mainly caused by the position and geometry of the lens or mirror, so that light rays with different incident angles are focused at different focal points, resulting in blurred imaging; coma is caused by the asymmetric processing of off-axis light rays by the optical system, which is related to the shape and configuration of the lens or mirror, and makes the convergence point of the light rays deviate from the ideal focal plane, resulting in the formation of coma. Since the factors of the three aberration phenomena are different, the corresponding imaging pictures are visually significantly different. The prior art needs to rely on manual judgment of the aberration types of a large number of images obtained by the imaging system before correcting aberrations, resulting in large errors. Summary of the Invention

[0004] In view of this, the present invention aims to provide an intelligent aberration detection method for space optical remote sensing payloads. By introducing a multi-head depthwise separable convolutional attention mechanism (Multi-DConv-HeadAttention, MDHA) on the basis of the original YOLOv8 network architecture, it realizes the automatic detection of star point targets in optical remote sensing images, and can efficiently deduce the aberration situation of the imaging system, thereby providing key information support for the in-orbit aberration adjustment of optical remote sensing payloads.

[0005] To achieve the above object, the technical solution of the present invention is realized as follows:

[0006] An intelligent aberration detection method for space optical remote sensing payloads, comprising the following steps:

[0007] S1: Obtain a remote sensing image dataset, and preprocess the remote sensing image dataset to obtain a training set;

[0008] S2: Construct a YOLOv8_MA network architecture, and use the training set obtained in step S1 to train the YOLOv8_MA network architecture to obtain a YOLOv8_MA model; The YOLOv8_MA network architecture is based on the YOLOv8 network architecture, and an MDHA module and a C2f_MA module with an MDHA sub-module are added to the neck network of the YOLOv8 network architecture;

[0009] S3: Input the remote sensing image to be detected into the YOLOv8_MA model obtained in step S2 for classification detection to obtain the corresponding aberration category.

[0010] Further, step S1 includes:

[0011] S11: Classify the aberration categories of the remote sensing images in the remote sensing image dataset, and label the remote sensing images as four aberration categories: spherical aberration, coma aberration, astigmatism, and no aberration;

[0012] S12: Divide the remote sensing images into blocks, and label the divided image blocks according to the four aberration categories labeled in step S11;

[0013] S13: Perform data augmentation on the image blocks obtained in step S12, and label the aberration categories of the data-augmented image blocks to obtain a training set.

[0014] Further, in step S2, the neck network of the YOLOv8_MA network architecture further includes a C2f module, a CBS module, and an upsampling module; the feature map B1 output by the backbone network of the YOLOv8_MA network architecture is processed by the MDHA module to obtain the feature map A1; the feature map A1 is upsampled by the upsampling module and then concatenated with the feature map B2 output by the backbone network to obtain the feature map A2; the feature map A2 is successively processed by the C2f module to obtain the feature map A3; the feature map A3 is upsampled by the upsampling module and then concatenated with the feature map B3 output by the backbone network to obtain the feature map A4; the feature map A4 is processed by the C2f_MA module to obtain the feature map A5; after the operation of the CBS module on the feature map A5, it is concatenated with the feature map A3 to obtain the feature map A6; the feature map A6 is processed by the C2f_MA module to obtain the feature map A7; after the operation of the CBS module on the feature map A7, it is concatenated with the feature map A1 to obtain the feature map A8; the feature map A8 is processed by the C2f_MA module to obtain the feature map A9, and the feature maps A5, A7, and A9 are all input into the head network of the YOLOv8_MA network architecture.

[0015] Further, in the C2f_MA module: the input feature is input into the CBS sub-module, the output feature of the CBS sub-module is split, and some of the split features are processed by a series of consecutive Bottleneck sub-modules. The features processed by each Bottleneck sub-module are concatenated with the two parts of the split features in terms of channels. The concatenated features are then processed by the CBS sub-module and enter the MDHA sub-module to obtain the output feature.

[0016] Further, step S3 includes the following steps:

[0017] S31: The remote sensing image to be detected is processed in blocks to obtain a plurality of image blocks;

[0018] S32: The image blocks obtained in step S31 are input into the YOLOv8_MA model for aberration category detection;

[0019] S33: Statistically analyze the aberration categories corresponding to all the image blocks to determine the aberration category of the remote sensing image to be detected.

[0020] Further, in step S33, analyze the proportion of the aberration categories corresponding to all the image blocks, and take the aberration categories with a proportion exceeding 50% of the threshold as the aberration categories of the remote sensing image to be detected.

[0021] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0022] In the intelligent aberration detection method for space optical remote sensing payloads of the present invention, the MDHA module and the C2f_MA module with MDHA sub-modules are placed in the neck network of the original YOLOv8. By using the MDHA algorithm to obtain the interdependence relationship of global context information, the attention to small targets is enhanced, the training speed and inference speed are improved, and at the same time, the target detection accuracy is also improved. The differences between several different aberration categories are effectively obtained, and the automated detection of star point targets in optical remote sensing images is realized, and the aberration situation of the imaging system is efficiently deduced, so as to provide key information support for the in-orbit aberration adjustment of optical remote sensing payloads. Description of the Drawings

[0023] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0024] Figure 1 It is a schematic flow chart of the intelligent aberration detection method for space optical remote sensing payloads described in the embodiments of the present invention;

[0025] Figure 2 It is a schematic diagram of four types of aberrations described in the embodiments of the present invention;

[0026] Figure 3 It is a schematic diagram of the YOLOv8_MA network architecture described in the embodiments of the present invention;

[0027] Figure 4 It is a schematic diagram of the C2f_MA module described in the embodiments of the present invention;

[0028] Figure 5 It is a schematic diagram of the MDHA module or MDHA sub-module described in the embodiments of the present invention. Detailed Embodiments

[0029] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following further describes the present invention in detail with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation to the present invention.

[0030] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0031] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise stated, the meaning of "a plurality" is two or more.

[0032] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "mounted", "connected", "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific situations.

[0033] The present invention will be described in detail below with reference to the drawings and in conjunction with embodiments.

[0034] As Figure 1 shown, the method for intelligent detection of aberrations of a space optical remote sensing payload according to an embodiment of the present invention includes the following steps:

[0035] S1: Obtain a remote sensing image data set and preprocess the remote sensing image data set to obtain a training set.

[0036] In some embodiments, step S1 includes:

[0037] S11: Classify the types of aberrations of the remote sensing images in the remote sensing image data set, and label the remote sensing images as four types of aberration categories: spherical aberration, coma, astigmatism, and no aberration respectively. As Figure 2 shown, Figure 2 in (a) shows a remote sensing image without aberration, Figure 2 in (b) shows a remote sensing image with spherical aberration, Figure 2 in (c) shows a remote sensing image with coma, Figure 2 in (d) shows a remote sensing image with astigmatism.

[0038] In one embodiment, a self-built remote sensing image dataset is adopted, and the images in this dataset are sourced from the remote sensing images captured by the payload for star imaging.

[0039] S12: To effectively process large remote sensing images captured by on-orbit remote sensing payloads and considering that these remote sensing images may contain numerous star targets, directly processing the entire remote sensing image would lead to excessive memory occupancy and reduced computational efficiency, and may also miss those smaller target areas. Therefore, the large remote sensing image is divided into blocks, and the divided image blocks are labeled according to the four aberration categories labeled in step S11.

[0040] In one embodiment, the entire remote sensing image is divided into multiple smaller image blocks through a window of size n×n. The specific size of the window, the division step size, and whether the size of the window changes during the sliding segmentation can all be adjusted according to actual needs. In a specific embodiment, the size of the window and the sliding step size are fixed, that is, the size is always 512×512 and the sliding step size is always 256. It can be understood that the sizes of the divided image blocks are consistent, all being 512×512, and the sliding segmentation is performed with a step size of 256. The entire remote sensing image before segmentation corresponds to one aberration category, so the aberration categories corresponding to the divided image blocks should also be the same as the aberration category corresponding to the entire remote sensing image before segmentation. And in one embodiment, the divided image blocks are manually labeled.

[0041] S13: The image blocks obtained in step S12 are subjected to data augmentation, and the aberration category labels are added to the image blocks after data augmentation to obtain a training set.

[0042] In some embodiments, the data augmentation operations include data type conversion, normalization processing, and quantization processing to improve the computational processing efficiency, optimize memory usage, and ensure that small target areas are not overlooked. At the same time, the data augmentation operations themselves do not change the aberration category of the image to be processed. Therefore, the aberration categories of the image blocks after data augmentation are the same as those of the image blocks before data augmentation. The training set obtained at this time includes: the image blocks of the remote sensing images, and the corresponding aberration categories of each image block.

[0043] In one embodiment, the image blocks are converted from the tif format to the png format to improve the subsequent model processing speed; the normalization processing is to enhance the contrast of the star areas through adaptive histogram equalization; the quantization processing is to compress the bit depth of the image (from 10 bits to 8 bits).

[0044] S2: Construct the YOLOv8_MA network architecture and train the YOLOv8_MA network architecture using the training set obtained in step S1 to obtain the YOLOv8_MA model. In one embodiment, since the image patches containing remote sensing images and the training set of aberration categories corresponding to each image patch have been obtained in step S1, the image patches of remote sensing images are used as the input for training, and the aberration categories corresponding to the image patches are used as the ground truth labels to train the YOLOv8_MA network architecture.

[0045] The YOLOv8_MA network architecture provided by the present invention is based on the YOLOv8 network architecture, and the MDHA mechanism (described in the paper "Primer: Searching for Efficient Transformers for Language Modeling" published on the arXiv platform) is added to the neck network of the YOLOv8 network architecture. The MDHA mechanism can focus on small objects or difficult-to-detect areas in the feature map, improve the detection accuracy of small objects, and can also establish connections between features of different scales to promote feature fusion, so as to better utilize information at different levels, thereby effectively solving the problem of neglecting the connection between the global context information of the image in the YOLOv8 network architecture.

[0046] Specifically, the YOLOv8_MA network architecture is as Figure 3 shown, and also includes a backbone network, a neck network, and a head network.

[0047] The backbone network has the same structure as the backbone network in the YOLOv8 network architecture, including C2f modules, CBS modules, and SPPF modules. The input image Input to the backbone network is processed successively by two cascaded CBS modules, C2f modules, CBS modules, and C2f modules to obtain the feature map B3; the feature map B3 is processed successively by a CBS module and a C2f module to obtain the feature map B2; the feature map B2 is processed successively by a CBS module, a C2f module, and an SPPF module to obtain the feature map B1.

[0048] The neck network also includes an MDHA module, a C2f_MA module with an MDHA sub-module, a C2f module, a CBS module, and an upsampling module. After the feature map B1 output by the backbone network is processed by the MDHA module, the feature map A1 is obtained; after the feature map A1 is upsampled by the upsampling module, it is concatenated with the feature map B2 output by the backbone network in the channel dimension to obtain the feature map A2; after the feature map A2 is sequentially processed by the C2f module, the feature map A3 is obtained; after the feature map A3 is upsampled by the upsampling module, it is concatenated with the feature map B3 output by the backbone network in the channel dimension to obtain the feature map A4; after the feature map A4 is processed by the C2f_MA module, the feature map A5 is obtained; after the operation of the CBS module on the feature map A5, it is concatenated with the feature map A3 in the channel dimension to obtain the feature map A6; after the feature map A6 is processed by the C2f_MA module, the feature map A7 is obtained; after the operation of the CBS module on the feature map A7, it is concatenated with the feature map A1 in the channel dimension to obtain the feature map A8; after the feature map A8 is processed by the C2f_MA module, the feature map A9 is obtained. The feature maps A5, A7, and A9 are all input into the head network.

[0049] The head network has the same structure as the head network structure in the YOLOv8 network architecture, including 3 detection heads. The feature maps A5, A7, and A9 are correspondingly input into the 3 detection heads for processing to obtain the aberration categories corresponding to the image Input.

[0050] In the present invention, the MDHA mechanism is placed at the input position of the neck network in the original YOLOv8 network architecture, and the C2f_MA replaces the last three C2f modules in the neck network. By using the MDHA mechanism to obtain the global context information interdependence relationship, the attention to small targets is enhanced, the training speed and the inference speed are improved, and at the same time, the object detection accuracy is also improved. The differences between several different aberration categories can be effectively obtained, and the model detection accuracy and efficiency are improved.

[0051] Specifically, the structure of the C2f_MA module is as Figure 4 shown. The input feature is input into the CBS sub-module, the output feature of the CBS sub-module is split, and the partial features obtained by splitting are processed by a series of consecutive Bottleneck sub-modules. Then, the features obtained by processing each Bottleneck sub-module are concatenated with the two parts of the features obtained by splitting in the channel dimension. After the features after channel concatenation are processed by the CBS sub-module, they enter the MDHA sub-module to obtain the output feature. In an embodiment, the output feature of the CBS sub-module is split into two features with the same number of channels according to the channel dimension. The structures of the CBS sub-module and the CBS module are the same as the structure of the CBS module in the YOLOv8 network architecture. The structures of the MDHA module and the MDHA sub-module are the same and are correspondingly constructed based on the publicly disclosed MDHA mechanism, asFigure 5 As shown, first, the input features generate Query (Q), Key (K), and Value (V) through three different linear transformations. Then, K, Q, and V are processed by three 3×1 depthwise separable convolutional kernels, achieving feature extraction, dimension transformation, non-linear mapping, information fusion, and reducing computational complexity. This can improve the model training and inference speed, and further contribute to enhancing the effect of the self-attention mechanism and improving the model's performance ability. In MDHA, the attention weights are finally calculated, and the output is transformed into the original dimension or the target dimension through the feed-forward network of the self-attention mechanism.

[0052] In the process of training the YOLOv8_MA network architecture to obtain the YOLOv8_MA model, the loss function used is the same as that for training the YOLOv8 network architecture.

[0053] In one embodiment, the AdamW optimizer is used to train the YOLOv8_MA network architecture. The initial learning rate during training is 0.001, and the weight decay is set to 10 -4 ; beta1 (the decay rate for calculating the exponential moving average of the gradient) is default set to 0.9, beta2 (the decay rate for calculating the exponential moving average of the squared gradient) is default set to 0.999, the Batch Size is set to 8, and the Epoch is set to 100.

[0054] S3: Input the remote sensing image to be detected into the YOLOv8_MA model obtained in step S2 for classification detection to obtain the corresponding aberration category.

[0055] In some embodiments, step S3 includes the following steps:

[0056] S31: Process the remote sensing image to be detected in blocks to obtain multiple image blocks;

[0057] S32: Input the image blocks obtained in step S31 into the YOLOv8_MA model for aberration category detection;

[0058] S33: Statistically analyze the aberration categories corresponding to all image blocks to determine the aberration category of the remote sensing image to be detected.

[0059] In one embodiment, since a remote sensing image is divided into multiple pictures for aberration detection, multiple detection results can be obtained for one remote sensing image. Therefore, in step S33, analyze the proportion of aberration categories corresponding to all image blocks, and use the aberration categories with a proportion exceeding 50% of the threshold as the aberration categories of the remote sensing image to be detected. The threshold can be selected manually or adaptively according to the actual situation. It is acceptable to use conventional human experience or multiple human experiments to adjust to obtain the threshold, as well as existing algorithms for adaptive threshold calculation. The present invention does not limit this.

[0060] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the disclosure of the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in the present invention can be achieved. No limitation is made herein.

[0061] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent detection method for aberration of a space optical remote sensing payload, characterized in that It includes the following steps: S1: Obtain a remote sensing image dataset, and preprocess the remote sensing image dataset to obtain a training set; S2: Construct a YOLOv8_MA network architecture, and use the training set obtained in step S1 to train the YOLOv8_MA network architecture to obtain a YOLOv8_MA model; the YOLOv8_MA network architecture uses the YOLOv8 network architecture as the basic network, and adds an MDHA module and a C2f_MA module with an MDHA sub-module to the neck network of the YOLOv8 network architecture; The neck network of the YOLOv8_MA network architecture further includes a C2f module, a CBS module, and an upsampling module; the feature map B1 output by the backbone network of the YOLOv8_MA network architecture is processed by the MDHA module to obtain a feature map A1; the feature map A1 is upsampled by the upsampling module and then concatenated with the feature map B2 output by the backbone network to obtain a feature map A2; the feature map A2 is sequentially processed by the C2f module to obtain a feature map A3; the feature map A3 is upsampled by the upsampling module and then concatenated with the feature map B3 output by the backbone network to obtain a feature map A4; the feature map A4 is processed by the C2f_MA module to obtain a feature map A5; the feature map A5 is operated by the CBS module and then concatenated with the feature map A3 to obtain a feature map A6; the feature map A6 is processed by the C2f_MA module to obtain a feature map A7; the feature map A7 is operated by the CBS module and then concatenated with the feature map A1 to obtain a feature map A8; the feature map A8 is processed by the C2f_MA module to obtain a feature map A9, and the feature map A5, the feature map A7, and the feature map A9 are all input into the head network of the YOLOv8_MA network architecture; S3: Input the remote sensing image to be detected into the YOLOv8_MA model obtained in step S2 for classification detection to obtain the corresponding aberration category.

2. The intelligent aberration detection method for space optical remote sensing payload according to claim 1, wherein Step S1 includes: S11: Classify the aberration categories of the remote sensing images in the remote sensing image dataset, and label the remote sensing images as four aberration categories: spherical aberration, coma aberration, astigmatism, and no aberration; S12: Divide the remote sensing images into blocks, and label the divided image blocks according to the four aberration categories labeled in step S11; S13: Perform data augmentation on the image blocks obtained in step S12, and perform aberration category labeling on the image blocks after data augmentation to obtain the training set.

3. The intelligent aberration detection method for space optical remote sensing payload according to claim 1, characterized in that In the C2f_MA module: Input the input features into the CBS sub-module, split the output features of the CBS sub-module, after processing some of the split features through a series of consecutive Bottleneck sub-modules, the features processed by each Bottleneck sub-module are concatenated with the two parts of the split features in channels, and the concatenated features are processed by the CBS sub-module and then enter the MDHA sub-module to obtain the output features.

4. The intelligent aberration detection method for space optical remote sensing payload according to claim 1, characterized in that, Step S3 includes the following steps: S31: Process the remotely sensed image to be detected in blocks to obtain multiple image blocks; S32: Input the image blocks obtained in step S31 into the YOLOv8_MA model for aberration category detection; S33: Statistically analyze the aberration categories corresponding to all the image blocks to determine the aberration category of the remotely sensed image to be detected.

5. The intelligent aberration detection method for space optical remote sensing payload according to claim 4, characterized in that In step S33, analyze the proportion of the aberration categories corresponding to all the image blocks, and take the aberration categories with a proportion exceeding 50% of the threshold as the aberration categories of the remotely sensed image to be detected.

Images