Large-area remote sensing image mosaic chromatic aberration automatic detection method

Abstract

The present application belongs to the technical field of optical remote sensing image processing, and specifically relates to a "large-area remote sensing image mosaic color difference automatic detection method". Through data set construction, neural network model deployment and model training, a classification detection method based on a convolutional neural network is used to perform color difference reasoning on image blocks cut along a mosaic line, and color difference image blocks are labeled. Then, according to the density of color difference points corresponding to the color difference image blocks, whether the number of continuous color difference points exceeds a threshold value is determined through a sliding window strategy, and then a color difference vector region is labeled. The present application can batch process all images in a full survey area, greatly reduce the time of manual participation, and automatically generate a labeled color difference region vector, thereby greatly improving the remote sensing image mosaic color difference detection efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of optical remote sensing image processing, and particularly relates to a mosaic color difference detection technology for large-area remote sensing images. BACKGROUND

[0002] Remote sensing image high-level mosaic products have important application value in environmental detection, disaster assessment and emergency management, and the fields of agriculture and forestry. In the generation process of remote sensing image high-level mosaic products, the splicing of original images along the mosaic line is a key step, mainly because the ground area that can be covered by a single remote sensing satellite sensor in one transit is limited, and the demand of users for remote sensing data often covers a wider area. In order to form a continuous image map covering a larger geographical range, multiple original images obtained by the same sensor at different times or images obtained by different sensors need to be spliced. In this process, the mosaic line is the boundary line of the splicing of each image, and splicing along these mosaic lines can ensure the continuity and consistency of the image.

[0003] However, the process of splicing along the mosaic line may cause color difference problems on both sides of the mosaic line. In the production and quality inspection process of large-area remote sensing image high-level products, a reasonable way is needed to quickly detect the mosaic color difference. Manual detection and vector labeling will consume a lot of manpower, so it is necessary to explore a reasonable and fast automatic detection method for mosaic color difference to meet the demand of mass remote sensing image mosaic splicing and quality inspection.

[0004] The existing automatic detection method for mosaic color difference includes image classification technology based on neural network. The new generation of convolutional neural network ConvNeXt model captures global dependency through large receptive field and uses large kernel convolution as the main building block. The latest part of the Transformer network is introduced into the existing modules of ConvNeXt to combine the advantages of the two networks and improve the overall performance of the model. For example, the literature entitled "Blind Evaluation Method for Remote Sensing Image Mosaic Quality Based on Weak Supervision" published in the Journal of Beijing University of Aeronautics and Astronautics, Vol. 49, No. 9, 2518-2526, 2022, discloses a double-branch network to process the color difference on both sides of the mosaic line in remote sensing images, and a set of simulated mosaic data sets are constructed to train the network.

[0005] However, the above method can only evaluate a single small image, and the data set size is only tens of thousands, which cannot process a large amount of remote sensing data and thus cannot be applied to the mosaic color difference detection of large-area remote sensing images. The performance of the simulated mosaic data set cannot be compared with the manually labeled data in real scenarios. In addition, it is also impossible to generate problem vectors for the color difference abnormal areas near the mosaic line based on the image detection results. SUMMARY

[0006] In order to deal with the diversity and complexity of remote sensing images in the generation process of high-level mosaic products of remote sensing images, the problem vector generation of the color difference abnormal area near the mosaic line cannot be carried out, and the existing algorithm cannot process a large amount of remote sensing data. The present application provides a "large-area remote sensing image mosaic color difference automatic detection method".

[0007] The large-area remote sensing image mosaic color difference automatic detection method, as shown in Figure 1 The steps include the following steps:

[0008] S1, data set construction: collect artificial annotation data, cut the mosaic line color difference vector annotated by artificial annotation into a plurality of map blocks as color difference samples, and collect a plurality of color difference free samples as neural network model training data set;

[0009] S2, neural network model deployment and model training: the neural network model adds a Stem module in the ConvNeXt model as an entry part, which is responsible for processing the input image and preparing the feature map for the subsequent network layer, the subsequent stage is composed of the ConvNeXt model every stage module according to the distribution ratio 3:3:9:3, and the GRN is introduced between the convolution layers of the ConvNeXt model, after the neural network model is deployed, the neural network model training data set is applied to train;

[0010] S3, inference of color difference positive sample image and labeling of color difference point: the neural network model infers the full survey area image cut along the mosaic line, so as to complete the classification of color difference image block and color difference free image block, and obtain the classified image block, and the inferred color difference positive sample image is labeled as color difference point;

[0011] S4, determine the color difference vector area by sliding window strategy: each image block in S3 corresponds to a geographic center point, for the image block determined as color difference image block, the center point is defined as color difference point, otherwise as color difference free point, through the determination of the proportion of the number of continuous color difference points in the unit of mosaic line, if the proportion exceeds the threshold value, the color difference area is determined, the specific determination strategy is traversed in the way of sliding window algorithm, and the high proportion color difference point area is marked, and then the high proportion color difference point area is vector expanded, so as to obtain the required color difference vector, so as to complete the automatic labeling of mosaic color difference, and realize the automatic detection of large-area remote sensing image mosaic color difference through batch processing of full survey area image.

[0012] Technical effects:

[0013] The present application first proposes an automatic detection method for "large-area" remote sensing image mosaic color difference, as shown in Figures 2-4 The mosaic color difference detection effect diagram after the present application is selected in part areas, Figure 2The region belongs to a plain region with good original data quality, and the color difference problem is less; Figure 3 The color difference problem occurs occasionally due to terrain restrictions in the region; Figure 4 The region belongs to a difficult area with complex terrain, and the color difference problem is more; the color difference vector determination strategy is proposed combined with neural network reasoning, and then the color difference area is labeled, and the labeled color difference area vector is automatically generated, so as to control the final product quality of remote sensing image, save labor cost, improve product delivery speed, improve the timeliness and response speed of large-area remote sensing image in disaster assessment, satellite image processing, and agricultural and forestry economic management fields, and improve the detection efficiency of remote sensing image mosaic color difference.

[0014] From the existing batch normalization to layer normalization, the application of layer normalization makes the neural network model more stable during training, which helps to improve the generalization ability of the model, and is convenient for processing the diversity and complexity of remote sensing images; as a pure CNN architecture, it combines the advantages of attention-based classifiers and traditional ResNet architectures, can batch process all images in the test area, and assist the global massive artificial annotation dataset related to the "Jilin No. 1" satellite, the dataset size reaches millions, greatly improves the determination accuracy, and solves the technical problem that the existing algorithm cannot process a large amount of remote sensing data. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The whole flowchart of the present application is shown in the figure, and is used as an abstract drawing.

[0016] Figure 2 The effect diagram of the mosaic color difference detection in the plain area is shown in the figure.

[0017] Figure 3 The effect diagram of the mosaic color difference detection in the terrain restricted area is shown in the figure.

[0018] Figure 4 The effect diagram of the mosaic color difference detection in the terrain restricted area is shown in the figure. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely in the embodiments of the present application combined with the drawings, obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments, by using the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the present application.

[0020] S1, dataset construction: first, a large amount of artificial annotation data in the "Jilin No. 1" dataset is collected, which covers various terrains around the world. By cropping the mosaic line color difference vector annotated by artificial annotation, multiple map blocks are cropped according to the size of 224x224 pixels, 100,000 color difference samples are obtained, and 3 million non-color difference samples are collected as neural network model training dataset, of which 80% is divided into training set and 20% is divided into verification set.

[0021] S2, neural network model deployment and model training: on the basis of the original ConvNeXt model architecture, a Stem module is added, which adopts the "Patchify" design. This design can effectively decompose the input image into smaller image blocks, and the subsequent stage is composed of ConvNeXt model each stage module in the proportion of 3:3:9:3, which ensures that the network can effectively process features at different levels. The introduction of GRN enhances the extraction ability of global features, thereby further improving the performance of the model in visual tasks; each stage module is composed of a layer of deep convolution and a subsequent 1x1 convolution layer. Deep convolution groups the input channels to achieve the effect of grouped convolution, which can reduce the number of model parameters and improve computational efficiency, while the combined 1x1 convolution layer is used to integrate the features in each group.

[0022] The GRN in S2 is composed of global feature aggregation, feature normalization and feature calibration, specifically:

[0023] Given an input feature X∈R H×W×C , a spatial feature X i is mapped to a vector gx:

[0024] G(X):=X∈R H×W×C →gx∈R C ,

[0025] Using L2 norm-based feature aggregation, a set of aggregation values is obtained,

[0026] G(X)=gx={||X1||,||X2||,…,||X C ||}∈R C , where G(X) i =||X i || is a scalar that aggregates the statistical quantity of the i-th channel. Next, a response normalization function N(·) is applied to the aggregation value, which uses the following standard split normalization:

[0027] where ||X i|| is the L2 norm of the i-th channel, and finally, the original input responses are calibrated using the computed feature normalization scores:

[0028] X i = X i * N(G(X) i ) ∈ R H×W .

[0029] The collected 3.1 million positive and negative chromatic aberration sample blocks are used for training, all the maps are cut along the mosaic line into 224*224 map blocks, and the neural network model training specific parameters are as follows: the learning rate is set to 1e-4, the weight decay is set to 1e-3, the batch size is set to 256, the training number is 100 times of traversing the whole data set, and the network with the highest verification set accuracy is taken as the final pre-training network.

[0030] The preferred embodiments of the application are described in detail above with reference to the drawings, but the application is not limited to the specific details in the above-described embodiments, and various simple modifications can be made to the technical solutions of the application within the technical concept of the application, and these simple modifications all belong to the protection scope of the application.

Claims

1. An automated method for detecting color difference in mosaic patterns of large-area remote sensing images, characterized in that: Includes the following steps: S1. Dataset Construction: Collect manually labeled data, crop the color difference vectors of the mosaic lines labeled by the manual labeling into multiple map blocks as color difference samples, and collect a number of color difference-free samples as a training dataset for the neural network model. S2. Neural Network Model Deployment and Training: The neural network model adds a Stem module to the ConvNeXt model as the entry point, which is responsible for processing the input image and preparing feature maps for subsequent network layers. The subsequent stages are composed of modules of each stage of the ConvNeXt model in a distribution ratio of 3:3:9:

3. GRN is introduced between the convolutional layers of the ConvNeXt model. After the neural network model is deployed, it is trained using the neural network model training dataset. S3. Reasoning for positive color difference sample images and labeling color difference points: The neural network model reasons on the full test area image after cropping along the mosaic line, thereby completing the classification of color difference image blocks and non-color difference image blocks, obtaining well-classified image blocks, and the reasoned positive color difference sample images will be labeled as color difference points; S4. Determine the color difference vector region using a sliding window strategy: In S3, each image patch corresponds to its geographic center point. For an image patch identified as a color difference patch, its center point is defined as a color difference point; otherwise, it is considered to have no color difference point. The color difference region is determined by the proportion of consecutive color difference points in units of mosaic lines. If the proportion exceeds a threshold, the color difference region is determined. The specific determination strategy is to traverse the image using a sliding window algorithm to mark the high proportion of color difference point regions. Then, the vector of these high proportion of color difference point regions is expanded outward to obtain the required color difference vector, thereby completing the automatic annotation of mosaic color difference. Automatic detection of mosaic color difference in large-area remote sensing images is achieved through batch processing of images of the entire survey area.

2. The automated detection method for mosaic chromatic aberration in large-area remote sensing images according to claim 1, characterized in that, S1 specifically involves: collecting a large amount of manually labeled data from the "Jilin-1" dataset; cropping the color difference vectors of the manually labeled mosaic lines into multiple image blocks of 224×224 pixels to obtain 100,000 color difference samples; and collecting 3 million color-difference-free samples as a training dataset for the neural network model, with 80% allocated to the training set and 20% to the validation set.

3. The automated detection method for mosaic color difference in large-area remote sensing images according to claim 1, characterized in that, In S2, each stage module consists of a deep convolutional layer and a subsequent 1x1 convolutional layer. The deep convolutional layer can achieve the effect of grouped convolution by grouping the input channels, and the 1x1 convolutional layer can integrate the features within each group.

4. The automated detection method for mosaic color difference in large-area remote sensing images according to claim 3, characterized in that, The specific parameters for training the neural network model in S2 are as follows: the learning rate is set to 1e-4, the weight decay is set to 1e-3, the batch size is set to 256, the training iterations cover the entire dataset 100 times, and the network with the highest accuracy on the validation set is taken as the final pre-trained network.

Images