Space optical remote sensing load aberration intelligent detection method

Abstract

The invention relates to the technical field of remote sensing images, in particular to a space optical remote sensing load aberration intelligent detection method, which comprises the following steps: introducing an MDHA mechanism into a YOLOv8 network to obtain a YOLOv8MA network, and carrying out aberration category classification detection on a remote sensing image by adopting a trained YOLOv8MA model so as to realize automatic detection on a star point target in an optical remote sensing image. The aberration condition of the imaging system can be efficiently deduced, so that key information support is provided for on-orbit aberration adjustment of the optical remote sensing load.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing images, and in particular relates to an intelligent detection method for aberration of space optical remote sensing payloads. Background Art

[0002] During the development of optical remote sensing payloads, their performance is affected by many factors, including ground gravity, vibration and shock during launch, and changes in the space gravity field and temperature after entering orbit. As the aperture and focal length of the optical system increase, the deformation of the optical system caused by factors such as gravity, vibration, shock and temperature may lead to large wavefront aberrations, which in turn affect the imaging performance of the remote sensor. To solve this problem, advanced space remote sensors have adopted active optical correction technology to improve imaging quality. The aberration adjustment of optical remote sensing payloads mainly adopts manual adjustment based on experience, which requires experienced operators to make adjustments. However, with the development of technology, image detection technology based on deep learning has also been gradually applied to the aberration adjustment of optical remote sensing payloads, which can more accurately capture and correct complex aberration phenomena.

[0003] Before active optical correction is performed, it is necessary to analyze the causes of aberrations in the imaging picture according to the type of aberration, so as to determine the correction position in the imaging system. Specifically, astigmatism is mainly caused by the difference between the state of the optical system and the ideal optical system, resulting in inconsistent focusing of light on different planes; spherical aberration is mainly caused by the position and geometry of the lens or reflector, causing light with different incident angles to focus at different focal points, resulting in blurred imaging; coma is caused by the asymmetric treatment of off-axis light by the optical system, which is related to the shape and configuration of the lens or reflector, causing the convergence point of the light to deviate from the ideal focal plane, leading to the formation of coma. Due to the different factors of the three aberration phenomena, the corresponding imaging pictures have obvious visual differences. The existing technology relies on manual interpretation of the aberration category of a large number of images obtained by the imaging system before correcting the aberration, which has large errors. Summary of the invention

[0004] In view of this, the present invention aims to provide an intelligent detection method for aberrations of space optical remote sensing payloads. Based on the original YOLOv8 network architecture, a multi-head deep separable convolutional attention mechanism (Multi-DConv-HeadAttention, MDHA) is introduced to realize automatic detection of star point targets in optical remote sensing images, and the aberration of the imaging system can be efficiently deduced, thereby providing key information support for the on-orbit aberration adjustment of the optical remote sensing payload.

[0005] To achieve the above object, the technical solution created by the present invention is implemented as follows: A space optical remote sensing payload aberration intelligent detection method comprises the following steps: S1: Obtain a remote sensing image dataset and preprocess the remote sensing image dataset to obtain a training set; S2: Build a YOLOv8_MA network architecture, and use the training set obtained in step S1 to train the YOLOv8_MA network architecture to obtain a YOLOv8_MA model; the YOLOv8_MA network architecture uses the YOLOv8 network architecture as the basic network, and adds an MDHA module and a C2f_MA module with an MDHA submodule to the neck network of the YOLOv8 network architecture; S3: Input the remote sensing image to be detected into the YOLOv8_MA model obtained in step S2 for classification detection to obtain the corresponding aberration category.

[0006] Further, step S1 includes: S11: classifying the aberration categories of the remote sensing images in the remote sensing image dataset, and marking the remote sensing images into four aberration categories: spherical aberration, coma, astigmatism, and no aberration; S12: Divide the remote sensing image into blocks, and mark the divided image blocks according to the four aberration categories marked in step S11; S13: performing data enhancement on the image blocks obtained in step S12, and labeling the aberration categories of the image blocks after data enhancement to obtain a training set.

[0007] Furthermore, in step S2, the neck network of the YOLOv8_MA network architecture further includes a C2f module, a CBS module, and an upsampling module; the feature map B1 output by the backbone network of the YOLOv8_MA network architecture is processed by the MDHA module to obtain the feature map A1; the feature map A1 is upsampled by the upsampling module and concatenated with the feature map B2 output by the backbone network to obtain the feature map A2; the feature map A2 is sequentially processed by the C2f module to obtain the feature map A3; the feature map A3 is upsampled by the upsampling module and concatenated with the feature map B3 output by the backbone network After the feature map A4 is processed by the C2f_MA module, the feature map A5 is obtained; after the feature map A5 is processed by the CBS module, the feature map A3 is concatenated to obtain the feature map A6; after the feature map A6 is processed by the C2f_MA module, the feature map A7 is obtained; after the feature map A7 is processed by the CBS module, the feature map A1 is concatenated to obtain the feature map A8; after the feature map A8 is processed by the C2f_MA module, the feature map A9 is obtained. The feature maps A5, A7 and A9 are all input into the head network of the YOLOv8_MA network architecture.

[0008] Furthermore, in the C2f_MA module: the input features are input into the CBS sub-module, the output features of the CBS sub-module are split, and the partial features obtained by the split are processed by multiple consecutive Bottleneck sub-modules. The features processed by each Bottleneck sub-module are concatenated with the two parts of the feature channels obtained by the split, and the spliced ​​features are processed by the CBS sub-module and then enter the MDHA sub-module to obtain the output features.

[0009] Furthermore, step S3 includes the following steps: S31: Divide the remote sensing image to be detected into blocks to obtain multiple image blocks; S32: Input the image block obtained in step S31 into the YOLOv8_MA model to perform aberration category detection; S33: Statistically analyzing the aberration categories corresponding to all image blocks to determine the aberration category of the remote sensing image to be detected.

[0010] Furthermore, in step S33, the proportions of aberration categories corresponding to all image blocks are analyzed, and the aberration categories whose proportions exceed 50% of the threshold are taken as the aberration categories of the remote sensing images to be detected.

[0011] Compared with the prior art, the invention can achieve the following beneficial effects: In the intelligent detection method for aberration of space optical remote sensing payload created by the present invention, an MDHA module and a C2f_MA module with an MDHA submodule are placed in the neck network of the original YOLOv8, and the MDHA algorithm is used to obtain the mutual dependence of global context information, thereby enhancing the focus on small targets, improving the training speed and the reasoning speed, and also improving the target detection accuracy, effectively obtaining the differences between several different aberration categories, realizing the automatic detection of star point targets in optical remote sensing images, and efficiently deducing the aberration of the imaging system, thereby providing key information support for the on-orbit aberration adjustment of the optical remote sensing payload. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings constituting part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments and descriptions of the present invention are used to explain the present invention and do not constitute an improper limitation on the present invention. In the drawings: Figure 1 A schematic diagram of a flow chart of a method for intelligently detecting aberrations of a space optical remote sensing payload according to an embodiment of the present invention; Figure 2 A schematic diagram of four aberrations described in an embodiment of the present invention; Figure 3 A schematic diagram of the YOLOv8_MA network architecture described in the embodiment of the present invention; Figure 4 A schematic diagram of a C2f_MA module according to an embodiment of the present invention; Figure 5 A schematic diagram of the MDHA module or MDHA submodule described in an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the invention more clear, the invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the invention and do not constitute a limitation of the invention.

[0014] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0015] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0016] In the description of the invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the invention can be understood according to specific circumstances.

[0017] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0018] like Figure 1 As shown, the space optical remote sensing payload aberration intelligent detection method described in the embodiment of the present invention comprises the following steps: S1: Obtain a remote sensing image dataset and preprocess the remote sensing image dataset to obtain a training set.

[0019] In some embodiments, step S1 includes: S11: Classify the aberration categories of the remote sensing images in the remote sensing image dataset, and mark the remote sensing images into four aberration categories: spherical aberration, coma, astigmatism, and no aberration, such as Figure 2 As shown, Figure 2 (a) shows a remote sensing image without aberration. Figure 2 (b) shows the remote sensing image of spherical aberration, Figure 2 (c) shows the remote sensing image of coma, Figure 2 (d) shows the remote sensing image of astigmatism.

[0020] In one embodiment, a self-built remote sensing image dataset is used, and the images in the dataset are derived from remote sensing images of star point imaging by a payload.

[0021] S12: In order to effectively process the large remote sensing images captured by the on-orbit remote sensing payload, and considering that these remote sensing images may contain many star point targets, directly processing the entire remote sensing image will result in excessive memory usage and reduced computational efficiency, and may also miss those smaller target areas. Therefore, the large remote sensing image is divided into blocks, and the divided image blocks are labeled according to the four aberration categories labeled in step S11.

[0022] In a certain embodiment, the entire remote sensing image is divided into multiple smaller image blocks through a window of size n×n, wherein the specific size of the window, the step length of the division, and whether the size of the window changes during the sliding segmentation process can be adjusted according to actual needs. In a specific embodiment, the size of the window and the sliding step length are fixed, that is, the size is always 512×512, and the sliding step length is always 256. It can be understood that the size of the image blocks obtained by segmentation is consistent, and they are all 512×512, and the sliding segmentation is performed with a step length of 256. The entire remote sensing image before segmentation corresponds to an aberration category, so the aberration category corresponding to the image block after segmentation should also be consistent with the aberration category corresponding to the entire remote sensing image before segmentation. And in a certain embodiment, the image blocks after segmentation are manually annotated.

[0023] S13: performing data enhancement on the image blocks obtained in step S12, and labeling the aberration categories of the image blocks after data enhancement to obtain a training set.

[0024] In some embodiments, the data enhancement operation includes conversion of data types, normalization processing and quantization processing, so as to improve the efficiency of computational processing, optimize memory usage, and ensure that weak target areas will not be ignored. At the same time, the data enhancement operation itself does not change the aberration category of the image to be processed, so the aberration category of the image block after data enhancement is consistent with the aberration category of the image block before data enhancement. The training set obtained at this time includes: the image block of the remote sensing image, and the aberration category corresponding to each image block.

[0025] In one embodiment, the image block is converted from tif format to png format to improve the subsequent model processing speed; the normalization process is to enhance the contrast of the star point area through adaptive histogram equalization; the quantization process is to compress the bit depth of the image (from 10 bits to 8 bits).

[0026] S2: Construct a YOLOv8_MA network architecture, and use the training set obtained in step S1 to train the YOLOv8_MA network architecture to obtain a YOLOv8_MA model. In one embodiment, since step S1 has obtained an image block containing a remote sensing image and a training set of aberration categories corresponding to each image block, the image block of the remote sensing image is used as the input for training, and the aberration category corresponding to the image block is used as the true label to train the YOLOv8_MA network architecture.

[0027] The YOLOv8_MA network architecture provided by the present invention uses the YOLOv8 network architecture as the basic network, and adds the MDHA mechanism (the paper "Primer: Searching for Efficient Transformers for Language Modeling" published on the arXiv platform) to the neck network of the YOLOv8 network architecture. The MDHA mechanism can focus on small objects or difficult-to-detect areas in the feature map, improve the detection accuracy of small objects, and establish connections between features of different scales to promote feature fusion, thereby better utilizing information at different levels, thereby effectively solving the problem of ignoring the connection between global context information of the image in the YOLOv8 network architecture.

[0028] Specifically, the YOLOv8_MA network architecture is as follows Figure 3 As shown, it also includes a trunk network, a neck network and a head network.

[0029] The backbone network is consistent with the backbone network structure in the YOLOv8 network architecture, including C2f module, CBS module and SPPF module. The image Input input to the backbone network is processed by two cascaded CBS modules, C2f modules, CBS modules and C2f modules in turn to obtain feature map B3; feature map B3 is processed by CBS modules and C2f modules in turn to obtain feature map B2; feature map B2 is processed by CBS modules, C2f modules and SPPF modules in turn to obtain feature map B1.

[0030] The neck network also includes an MDHA module, a C2f_MA module with an MDHA submodule, a C2f module, a CBS module, and an upsampling module. After the feature map B1 output by the backbone network is processed by the MDHA module, the feature map A1 is obtained; after the feature map A1 is up-sampled by the up-sampling module, the channel splicing operation is performed with the feature map B2 output by the backbone network to obtain the feature map A2; the feature map A2 is processed by the C2f module in turn to obtain the feature map A3; after the feature map A3 is up-sampled by the up-sampling module, the channel splicing operation is performed with the feature map B3 output by the backbone network to obtain the feature map A4; after the feature map A4 is processed by the C2f_MA module, the feature map A5 is obtained; after the feature map A5 is operated by the CBS module, the channel splicing operation is performed with the feature map A3 to obtain the feature map A6; after the feature map A6 is processed by the C2f_MA module, the feature map A7 is obtained; after the feature map A7 is operated by the CBS module, the channel splicing operation is performed with the feature map A1 to obtain the feature map A8; after the feature map A8 is processed by the C2f_MA module, the feature map A9 is obtained. Feature maps A5, A7, and A9 are all input into the head network.

[0031] The head network has the same structure as the head network in the YOLOv8 network architecture, including three detection heads. Feature maps A5, A7, and A9 are input into the three detection heads for processing to obtain the aberration category corresponding to the image Input.

[0032] The present invention puts the MDHA mechanism into the input position of the neck network in the original YOLOv8 network architecture, replaces the last three C2f modules in the neck network with C2f_MA, and uses the MDHA mechanism to obtain the mutual dependence of global context information, thereby enhancing the focus on small targets, improving the training speed and reasoning speed, and also improving the target detection accuracy, effectively obtaining the differences between several different aberration categories, and improving the model detection accuracy and efficiency.

[0033] Specifically, the structure of the C2f_MA module is as follows Figure 4As shown, the input features are input into the CBS submodule, the output features of the CBS submodule are split, and the split partial features are processed by a plurality of consecutive Bottleneck submodules, and then the features processed by each Bottleneck submodule are spliced ​​with the two parts of the split feature channels, and the spliced ​​channel features are processed by the CBS submodule and then enter the MDHA submodule to obtain the output features. In a certain embodiment, the output features of the CBS submodule are split into two features with the same number of channels according to the channel, and the structure of the CBS submodule and the CBS module is consistent with the structure of the CBS module in the YOLOv8 network architecture. The structure of the MDHA module is consistent with that of the MDHA submodule, and is constructed based on the above-disclosed MDHA mechanism, as shown in FIG. Figure 5 As shown in the figure, the input features are first transformed into Query (Q), Key (K) and Value (V) through three different linear transformations, and then K, Q and V are processed through three 3×1 depthwise separable convolution kernels, which realizes feature extraction, dimension transformation, nonlinear mapping, information fusion and reduces computational complexity, which can improve the model training and reasoning speed, and thus help improve the effect of the self-attention mechanism and improve the performance of the model. Finally, the attention weight is calculated in MDHA, and the output is converted to the original dimension or target dimension through the feedforward network of the self-attention mechanism.

[0034] In the process of training the YOLOv8_MA network architecture and obtaining the YOLOv8_MA model, the loss function used is consistent with the loss function used to train the YOLOv8 network architecture.

[0035] In one embodiment, the YOLOv8_MA network architecture is trained using the AdamW optimizer. The initial learning rate during training is 0.001 and the weight decay is set to 10. -4 ; beta1 (the decay rate of the exponential moving average used to calculate the gradient) is set to 0.9 by default, beta2 (the decay rate of the exponential moving average used to calculate the square of the gradient) is set to 0.999 by default, Batch Size is set to 8, and Epoch is set to 100.

[0036] S3: Input the remote sensing image to be detected into the YOLOv8_MA model obtained in step S2 for classification detection to obtain the corresponding aberration category.

[0037] In some embodiments, step S3 includes the following steps: S31: Divide the remote sensing image to be detected into blocks to obtain multiple image blocks; S32: Input the image block obtained in step S31 into the YOLOv8_MA model to perform aberration category detection; S33: Statistically analyzing the aberration categories corresponding to all image blocks to determine the aberration category of the remote sensing image to be detected.

[0038] In a certain embodiment, since a remote sensing image is divided into multiple pictures for aberration detection, a remote sensing image can obtain multiple detection results. Therefore, in step S33, the proportion of aberration categories corresponding to all image blocks is analyzed, and the aberration category with a proportion exceeding 50% of the threshold is used as the aberration category of the remote sensing image to be detected. The threshold is selected and adjusted manually or adaptively according to the actual situation. The threshold can be obtained by conventional human experience or multiple artificial experimental adjustments, as well as the existing threshold adaptive calculation algorithm. The present invention does not limit this.

[0039] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0040] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A space optical remote sensing payload aberration intelligent detection method, characterized in that: The following steps are involved: S1: Acquire a remote sensing image dataset, and preprocess the remote sensing image dataset to obtain a training set; S2: construct a YOLOv8_MA network architecture, and use the training set obtained in step S1 to train the YOLOv8_MA network architecture to obtain a YOLOv8_MA model; the YOLOv8_MA network architecture uses the YOLOv8 network architecture as a basic network, and adds an MDHA module and a C2f_MA module with an MDHA submodule to the neck network of the YOLOv8 network architecture; S3: Input the remote sensing image to be detected into the YOLOv8_MA model obtained in step S2 for classification detection to obtain the corresponding aberration category.

2. The space optical remote sensing payload aberration intelligent detection method according to claim 1, characterized in that: Step S1 includes: S11: classifying the remote sensing images in the remote sensing image data set into aberration categories, and marking the remote sensing images into four aberration categories: spherical aberration, coma, astigmatism, and no aberration; S12: Divide the remote sensing image into blocks, and mark the divided image blocks according to the four aberration categories marked in step S11; S13: performing data enhancement on the image blocks obtained in step S12, and labeling the aberration categories of the image blocks after data enhancement to obtain the training set.

3. The space optical remote sensing payload aberration intelligent detection method according to claim 1, characterized in that: In step S2, the neck network of the YOLOv8_MA network architecture further includes a C2f module, a CBS module and an upsampling module; the feature map B1 output by the backbone network of the YOLOv8_MA network architecture is processed by the MDHA module to obtain a feature map A1; the feature map A1 is upsampled by the upsampling module and concatenated with the feature map B2 output by the backbone network to obtain a feature map A2; the feature map A2 is sequentially processed by the C2f module to obtain a feature map A3; the feature map A3 is upsampled by the upsampling module and concatenated with the feature map B3 output by the backbone network to obtain a feature map feature map A4; after the feature map A4 is processed by the C2f_MA module, a feature map A5 is obtained; after the feature map A5 is operated by the CBS module, a splicing operation is performed with the feature map A3 to obtain a feature map A6; after the feature map A6 is processed by the C2f_MA module, a feature map A7 is obtained; after the feature map A7 is operated by the CBS module, a splicing operation is performed with the feature map A1 to obtain a feature map A8; after the feature map A8 is processed by the C2f_MA module, a feature map A9 is obtained, and the feature maps A5, A7 and A9 are all input into the head network of the YOLOv8_MA network architecture.

4. The space optical remote sensing payload aberration intelligent detection method according to claim 3, characterized in that: In the C2f_MA module: The input features are input into the CBS submodule, and the output features of the CBS submodule are split. After the split features are processed by multiple consecutive Bottleneck submodules, the features processed by each Bottleneck submodule are spliced ​​with the two parts of the split feature channels. The spliced ​​features are processed by the CBS submodule and then enter the MDHA submodule to obtain the output features.

5. The space optical remote sensing payload aberration intelligent detection method according to claim 1, characterized in that: Step S3 includes the following steps: S31: Divide the remote sensing image to be detected into blocks to obtain multiple image blocks; S32: Input the image block obtained in step S31 into the YOLOv8_MA model to perform aberration category detection; S33: Statistically analyzing the aberration categories corresponding to all image blocks to determine the aberration category of the remote sensing image to be detected.

6. The space optical remote sensing payload aberration intelligent detection method according to claim 5, characterized in that: In step S33, the proportions of aberration categories corresponding to all image blocks are analyzed, and the aberration categories whose proportions exceed 50% of the threshold are taken as the aberration categories of the remote sensing image to be detected.

Images