A lightweight semantic segmentation model remote sensing sample optimization method

Abstract

The present invention discloses a remote sensing sample optimization method for a lightweight semantic segmentation model, comprising the following steps: S1, collecting publicly available remote sensing semantic segmentation sample data on the Internet, sequentially performing data enhancement on the samples, constructing a lightweight semantic segmentation FCT network model, constructing a Beta hybrid model, establishing a label correction and refinement model, and optimizing the samples; S2, placing the high-quality samples selected in step S1 into a sample database, further screening and processing the samples according to the requirements of the incremental learning technology for the samples; S3, continuing training on the basis of the pre-trained model using the data set screened in step S2. The present invention can not only complete the rapid training of the model in the presence of a large number of samples, but also simultaneously achieve the purpose of correcting and refining the samples and removing redundant samples, while reducing the consumption of training resources without reducing the accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of AI sample data processing, and in particular to a remote sensing sample optimization method for a lightweight semantic segmentation model. Background Art

[0002] Deep learning is becoming increasingly important in the field of remote sensing data processing, but the particularity of remote sensing data, such as large data volume, high dimensionality, high annotation cost, and uneven sample distribution, poses challenges to the training and generalization capabilities of the model.

[0003] Existing deep learning semantic segmentation methods cannot effectively train and generalize the model when there is a lot of noise in the training samples, and the samples cannot be further optimized, making it difficult to obtain high-quality samples. At the same time, most models have many layers and complex network structures, which require a lot of computing resources and time costs. To address this problem, the present invention proposes a lightweight semantic segmentation model remote sensing sample optimization method, which can further denoise and optimize the samples while training the model, thereby improving the model accuracy and reducing computing resources and time costs. Summary of the invention

[0004] In order to solve the existing problems, the present invention provides a lightweight semantic segmentation model remote sensing sample optimization method, the specific scheme is as follows:

[0005] A lightweight semantic segmentation model remote sensing sample optimization method includes the following steps:

[0006] S1, collect publicly available remote sensing images and their corresponding classification labels on the Internet to form sample data for semantic segmentation, perform data enhancement on the samples, build a lightweight semantic segmentation FCT network model, build a Beta hybrid model, establish a label correction and refinement model, and optimize the samples;

[0007] S2, putting the high-quality samples selected in step S1 after correction and refinement into the sample database, and storing the pre-trained model into the model database, and further screening and processing the samples according to the requirements of the incremental learning technology for the samples;

[0008] S3, using the data set screened in step S2 to perform further training based on the pre-trained model.

[0009] Preferably, step S1 specifically includes the following steps:

[0010] S11, collect publicly available remote sensing images and their corresponding classification labels on the Internet to form sample data for semantic segmentation, and enter the data enhancement module. The data enhancement methods include random rotation, random scaling, random cropping, horizontal flipping, random erasing, image mixing, and color jittering;

[0011] S12, build a lightweight semantic segmentation FCT network model and use existing samples to pre-train the FCT network model;

[0012] S13, for the remote sensing images in all samples (including images and their corresponding labels), use the pre-trained model to calculate the LN score and RC score of the two quality assessment indicators, and use the Beta mixture model to model the two quality assessment index score results, and divide all samples into clean samples and noise samples at the image level;

[0013] S14, based on the above method, select clean samples to establish a label correction refinement model and optimize the samples.

[0014] Preferably, the lightweight semantic segmentation FCT network model constructed in step S12 specifically includes: an encoder module, a decoder module, an efficient transformation network module, a feature enhancement module and a pixel attention module;

[0015] The encoder module includes a lightweight dilated bottleneck layer (LDB) module, and the LDB structure consists of 1*1 convolution, 1*3 convolution and channel attention;

[0016] The efficient transformation network module uses the self-attention mechanism as the capsule network, and its formula is as follows:

[0017]

[0018] in, , , Value represent query vector, key vector and value vector respectively, represents the transpose of the key vector; D is the embedding dimension of the input feature layer; is the activation function, O is the normalized matrix;

[0019] The pixel attention (PA) module: The pixel attention mechanism learns weights according to the importance of features at different pixel positions, and the formula is:

[0020]

[0021] Where X is the output feature of the last dilated bottleneck layer in the decoder module, represents the sigmoid function, Indicates a convolution operation with a convolution kernel of 1*1. Represents the output features after the pixel attention mechanism operation;

[0022] The feature enhancement (FE) module performs feature dependency modeling from the channel dimension and the spatial dimension respectively.

[0023] Preferably, the construction of the Beta mixture model in step S13 specifically includes the following steps:

[0024] S131, for the remote sensing images in all samples (including images and their corresponding labels), the pre-trained model is used to output the probability map of the last layer of the network, and then the LN score and RC score of the two quality assessment indicators are calculated respectively; S132, the Beta mixture model is used to model the two quality assessment indicator score results, and all samples are divided into clean samples and noise samples at the image level.

[0025] Preferably, the label correction in step S14 refines the model and optimizes the sample, specifically including the following steps:

[0026] S141, in the clean sample and noise sample data sets, respectively calculate the distance weight and sample purity of each sample within the range of 100 adjacent samples around it, and the calculation formula is as follows:

[0027]

[0028] Where d is the Euclidean distance from the middle sample position to the 100 neighboring samples. is the average value of all distances, dw is the distance weight coefficient, when the middle sample is of the same type (clean, noisy) as the neighboring sample, dw is a positive number, otherwise it is a negative number, and pure is the purity score value.

[0029] S142, considering that only clean samples are used for subsequent modeling, only the purity of clean samples is selected. The spatial range covered by the clean samples is divided into geographical ranges, and the coordinates of the four corner points of the clean samples and the pure purity of the samples are calculated as features for clustering. The number of clusters is customized according to the image coverage, and the clustering method selects the Hierarchical clustering method.

[0030] S143, for the image data corresponding to the samples in each cluster after clustering, calculate the building index feature, vegetation index feature, water index feature and grayscale symbiotic texture matrix feature;

[0031] S144, after calculating the above features, first randomly obtain a number of points in each image label, and then if the number of label values ​​at the center point within a 5*5 range around the sampling point is greater than 80%, then retain this point as a training sample, and then use the spectral angle method to perform preliminary data cleaning;

[0032] S145, considering that the label accuracy in the training data may be low and affect the accuracy of the subsequent classification model, the samples are further denoised at the pixel level according to the process of estimating the joint distribution of noise labels and true labels, filtering noise samples, and training sample optimization model.

[0033] S146, using the model optimized by the trained samples to predict all samples, calculating the F1 score index for the optimized results and the original labels, performing a histogram analysis on the F1 score values, setting a threshold and selecting samples that meet the threshold conditions as high-quality samples.

[0034] Preferably, step S2 specifically includes the following steps:

[0035] S21, putting the samples selected in step S1 after correction and refinement into the sample database, and storing the pre-trained model into the model database;

[0036] S22, randomly select 25% of the screened optimized samples with high confidence in the sample database, and replace these samples with the original labels to update the entire training set.

[0037] Preferably, in step S3, the training set selected in step S2 is used to perform further training on the basis of the pre-trained model, and parameters such as the number of iterations, learning rate, activation function, etc. are set.

[0038] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. After the computer program is run, any of the above methods is executed.

[0039] The present invention also discloses a computer system, including a processor and a storage medium, wherein a computer program is stored on the storage medium, and the processor reads and runs the computer program from the storage medium to execute any of the methods described above.

[0040] The beneficial effects of the present invention are:

[0041] A lightweight semantic segmentation model is proposed to achieve fast model training in the presence of a large number of samples. Compared with non-lightweight models, lightweight semantic segmentation models have obvious advantages in computing resource consumption, inference speed, energy consumption, deployment flexibility, data efficiency and model interpretability. It is particularly suitable for real-time operation on resource-constrained devices, while reducing costs and energy consumption, and improving the practicality and flexibility of the model.

[0042] Two score indicators, LN score and RC score, are proposed to effectively identify sample quality. The pre-trained model trained in the early stage is used to calculate the two quality assessment score indicators of the remote sensing images in all samples. The Beta mixture model is used to model the two quality assessment score results. The samples can be divided into clean samples and noisy samples at the image level, and the samples with noise can be effectively identified from all samples.

[0043] A method is proposed to further identify noise pixels in samples and remove noise at the pixel level using confidence, which can provide good training data for subsequent pixel-based sample classification optimization and improve sample optimization accuracy.

[0044] The refined samples are used in combination with incremental learning technology to continue training the model, which improves the accuracy of the semantic segmentation model while reducing the training cost and consumption of computing resources.

[0045] The present invention can not only complete the rapid training of the model in the presence of a large number of samples, but also simultaneously achieve the purpose of correcting and refining the samples and removing redundant samples, while reducing the consumption of training resources without reducing the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 It is a flowchart of the method of the present invention;

[0048] Figure 2 This is a schematic diagram of the overall structure of the lightweight semantic segmentation FCT network;

[0049] Figure 3 This is a local detail diagram of the Encoder structure in the lightweight semantic segmentation FCT network;

[0050] Figure 4 This is a local detail diagram of the Feature Enhancement structure in the lightweight semantic segmentation FCT network. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] There are more and more studies on the application of remote sensing data in the field of deep learning. However, remote sensing data has problems such as large data volume, high dimensionality, high labeling cost, and uneven sample distribution, which pose challenges to the training and generalization capabilities of the model. Sample optimization is particularly important as a key step to improve model performance. It improves data quality through data enhancement and cleaning, reduces the need for manual labeling by using automated labeling tools and semi-supervised learning techniques, and solves the sample imbalance problem through methods such as resampling and generative adversarial networks, thereby improving the generalization ability and accuracy of the model.

[0053] In addition, the high-dimensional nature of remote sensing images requires the model to extract richer and more abstract features. Sample optimization improves the model's ability to capture key information by designing more effective network structures and feature fusion strategies, which is crucial for achieving more intelligent and automated remote sensing image analysis. Through technologies such as knowledge distillation and model pruning, sample optimization can also reduce the number of model parameters and computational complexity, achieving faster reasoning speed, which is particularly important for remote sensing application scenarios that require real-time processing.

[0054] In general, sample optimization plays a vital role in deep learning applications in the field of remote sensing. It not only improves the performance and efficiency of the model, but also reduces the dependence on a large amount of labeled data and improves the efficiency and quality of data processing. These optimization measures are of great value in promoting the further development and application of remote sensing technology, making remote sensing image analysis more accurate, efficient and intelligent.

[0055] In view of the low accuracy of sample labels in existing network datasets, the present invention proposes a remote sensing sample optimization method for a lightweight semantic segmentation model. First, the early samples are trained using the FCT lightweight network, and the LN and RC quality assessment index scores are calculated for the images in the samples. The Beta mixture model is used to model the two quality assessment index score results, and all samples are divided into clean samples and noise samples at the image level; then, the spectral angle method is used to preliminarily clean the samples in the clean sample data, and the confidence learning technology is further used in the classification model to further identify and remove noise samples at the pixel level; finally, the XGBoost algorithm is used to establish a label correction and refinement model; all samples are optimized, and the F1 score index is calculated for the optimized results and the original labels, and the threshold is set to select samples that meet the threshold conditions as high-quality samples.

[0056] Reference Figure 1 , Figure 1 : is a schematic diagram of the remote sensing image sample optimization process of the present invention, which includes the following steps:

[0057] S1: Collect remote sensing images and their corresponding classification labels to form sample data for semantic segmentation, perform data enhancement on the samples in turn, build a lightweight semantic segmentation FCT network model, build a Beta hybrid model, establish a label correction and refinement model, and optimize the samples.

[0058] Specifically,

[0059] S11, collects publicly available remote sensing images and their corresponding classification labels on the Internet to form sample data for semantic segmentation, and enters the data enhancement module. The data enhancement methods include random rotation, random scaling, random cropping, horizontal flipping, random erasing, image mixing and color jittering.

[0060] S12, build a lightweight semantic segmentation FCT network model, use existing samples to pre-train the FCT network model, and train it for 5-15 iterations (the specific number of training iterations is determined by the accuracy curve of the validation set, and the training is terminated if a sudden change occurs in the accuracy curve).

[0061] Among them, refer to Figure 2 It is a lightweight semantic segmentation network structure of the present invention, which specifically includes an encoder, a decoder, an efficient transformation network module, a feature enhancement module, and a pixel attention module, as follows:

[0062] The encoder is mainly composed of a lightweight dilated bottleneck layer (LDB) module. The LDB structure consists of 1*1 convolution, 1*3 convolution and channel attention. Figure 3 :

[0063] Partial details of the encoder:

[0064] First, at the bottleneck, the number of channels of the input feature is halved through 1*1 convolution. Then, in order to obtain a larger receptive field to capture a wider range of contextual information, 3*1 and 1*3 decomposition convolutions are added.

[0065]

[0066] In the formula, is the feature matrix of the network output before the expansion bottleneck layer, Indicates a convolution operation with a convolution kernel of 1*1.

[0067] Indicates a convolution operation with a convolution kernel of 3*1. Indicates a convolution operation with a convolution kernel of 1*3. is the output matrix after dimension compression;

[0068] Furthermore, in the next two branches, one uses decomposed convolution and the other uses dilated convolution;

[0069] Furthermore, the results of the two branches are processed through channel attention (CA) to enhance feature expression and suppress noise;

[0070]

[0071]

[0072] in, Indicates a decomposed convolution operation with a convolution kernel of 1*1. Indicates that the convolution kernel is a 1*1 dilated convolution operation. Indicates a decomposed convolution operation with a convolution kernel of 3*1. Indicates that the convolution kernel is a 3*1 dilated convolution operation. It is averaging in the channel dimension; is the output matrix after dimension compression, is the feature matrix output after the decomposition convolution operation, is the feature matrix output after the dilated convolution operation;

[0073] Furthermore, the low-dimensional branches and intermediate features after channel attention CA processing are fused and processed by 1*1 convolution to restore the number of channels to the same dimension as the input feature map;

[0074] Furthermore, the above output feature map is fused with the input feature map, and the fusion result is shuffled using the channel strategy to avoid the shortcomings of information independence and channel irrelevance caused by deep convolution and irrelevance, and to promote the exchange of semantic information between different channels. The formula is as follows:

[0075]

[0076] In the formula, The same as the input for calculating F1, which is the feature matrix of the network output before the expansion bottleneck layer, Indicates a convolution operation with a convolution kernel of 1*1. is the output matrix after dimension compression, is the feature matrix output after the decomposition convolution operation, is the feature matrix output after the dilated convolution operation, is the channel shuffle strategy, is the output matrix of the expanded bottleneck layer LDB module.

[0077] The efficient transformation network module uses the self-attention mechanism as the capsule network, and its formula is as follows:

[0078]

[0079] in, , , Value represent query vector, key vector and value vector respectively, represents the transpose of the key vector, and D is the embedding dimension of the input feature layer; is the activation function, O is the normalized matrix;

[0080] Pixel Attention (PA) mechanism: The pixel attention mechanism learns weights according to the importance of features at different pixel positions. The pixel attention mechanism pays more attention to the edges and textures of objects in the image. The calculation formula is as follows:

[0081]

[0082] Where X is the output feature of the last dilated bottleneck layer in the decoder module, represents the sigmoid function, Indicates a convolution operation with a convolution kernel of 1*1. Represents the output features after the pixel attention mechanism operation;

[0083] Feature Enhancement (FE): In order to fully integrate low-level spatial information and high-level semantic information, the feature enhancement FE module is designed to improve the ability of feature expression. Feature dependency modeling is performed from the channel dimension and spatial dimension respectively. The two dimensions are integrated to more effectively transfer low-level information to high-level. The specific structure is shown in Figure 4 , the local details of the feature enhancement module (FE) are as follows:

[0084] The input feature map is first average pooled and max pooled and dimensionally reduced in the two branches, respectively, with the following formula:

[0085]

[0086] in, is the output matrix of the expanded bottleneck layer LDB module, represents the sigmoid activation function, represents the ReLU activation function, B represents batch normalization, C is the number of channels of the feature map, r represents the dimensionality reduction factor, and Represent the average pooling and maximum pooling operations respectively, represents the output matrix of channel dimension, The output matrix representing the spatial dimensions, Indicates a convolution operation with a convolution kernel of 3*3. represents the feature concatenation operation, It represents a convolution operation with an output feature channel of C and a convolution kernel of 1*1. It represents a convolution operation with an output feature channel of C / r and a convolution kernel of 1*1.

[0087] Furthermore, the output results of the two branches are respectively convolved by 1*1 to restore the number of channels to the same as the input feature Figure 1 The formula is:

[0088]

[0089] in, is the output matrix of the expanded bottleneck layer LDB module, Indicates a convolution operation with a convolution kernel of 1*1. represents the output of the channel dimension, represents the output of the spatial dimension, is the output feature matrix after being processed by the feature enhancement module.

[0090] S13, construct a Beta hybrid model to model the two quality assessment index score results, and divide all samples into clean samples and noise samples at the image level; specifically, the following steps are included:

[0091] S131, for all remote sensing images in the samples (including images and their corresponding labels), the probability map of the last layer of the network is output by the pre-trained model, and then the LN score and RC score of the two quality assessment indicators are calculated respectively. Specifically, for a training sample (x, y), x represents the image data in the sample, y represents the label data in the sample, and the LN score and RC score of the two quality assessment indicators are calculated.

[0092] The calculation formula is as follows:

[0093] The LN score captures the impact of each sample on the gradient of the loss function. Specifically, for a training sample (x, y), its LN score is defined as the expected norm of the loss gradient at time t.

[0094]

[0095] Among them, x represents the image data in the sample, y represents the label data in the sample, are the model parameters at time t, is the predicted probability of the model for the kth category, K represents the number of categories, is the one-hot encoding of the true label, is the logit gradient for the kth category, Indicates the calculation of the square of the L2 norm, It means to calculate the mathematical expectation value. Indicates accumulation, Indicates the calculated LN score value at time t.

[0096] The RC score is defined as the expected value of the square of the L2 norm between the predicted probability vector and the true label vector. For a training sample (x, y), its RC score is expressed as:

[0097]

[0098] Among them, x represents the image data in the sample, y represents the label data in the sample, are the model parameters at time t, is the model's predicted output for the input x image data, is the one-hot encoding of the true label, Represents the square of the L2 norm, which calculates the square error between the predicted output and the true label. Represents the mathematical expectation of the L2 norm square of the predicted label and the true label at time t, Represents the calculated RCscore value at time t;

[0099] S132, the Beta mixture model is used to model the score results of the two quality assessment indicators, and all samples are divided into clean samples and noise samples at the image level.

[0100] S14, based on the above method, select clean samples to establish a label correction refinement model and optimize the samples.

[0101] The steps are:

[0102] S141, in the clean sample and noise sample data sets, respectively calculate the distance weight of each sample and its surrounding 100 adjacent samples, the weight selects a simplified Gaussian function, and then calculates the purity index:

[0103]

[0104]

[0105] Among them, i is the number of samples, d is the Euclidean distance from the center sample position to the surrounding 100 neighboring samples, is the average value of all distances, dw is the distance weight coefficient, when the center sample and the neighboring sample belong to the same type (clean, noisy), dw is a positive number, otherwise it is a negative number, and pure is the purity score value.

[0106] S142, considering that only clean samples are used in subsequent modeling, only the purity of clean samples is selected. The spatial range covered by the clean samples is divided into geographical ranges, and the coordinates of the four corner points of the clean samples and the purity of the samples are calculated and clustered together. The number of clusters is determined by the image coverage range, and the clustering method selects the Hierarchical clustering method.

[0107] S143, for the image data corresponding to the samples in each cluster after clustering, calculate the building index feature, vegetation index feature, water index feature and grayscale symbiotic texture matrix feature, as follows:

[0108] The above building index features: Normalized Difference Built-up Index (NDBI), Modified Building Index (MBI), Built-up and Water Index (BIWI), Built-up Vegetation Index (BVI), and their calculation formulas are as follows:

[0109] NDBI = (SWIR - NIR ) / ( SWIR + NIR)

[0110] MBI = SWIR / (SWIR + Red) - NIR / ( NIR + Red)

[0111] BIWI = SWIR - (Red + NIR)

[0112] BVI = Red - NIR / (Red + NIR)

[0113] The Red, NIR, and SWIR in the formula represent the red band, near infrared band, and short-wave infrared band, respectively.

[0114] The above vegetation index characteristics: Normalized Difference Vegetation Index (NDVI), Enhanced Vegetation Index (EVI), Modified Soil Adjusted Vegetation Index (MSAVI), Modified Difference Water Index (MDWI), Green Normalized Difference Vegetation Index (GNDVI), Ratio Vegetation Index (RVI), Simple Index of Photosynthetic Inefficiency (SIPI), the calculation formula is as follows:

[0115] NDVI = (NIR - Red) / (NIR + Red)

[0116] EVI = 2.5 * (NIR - Red ) / (NIR + 6 * Red - 7.5 * Blue + 1)

[0117] MSAVI = [(NIR - Red) / (NIR + Red + 0.16)] * 1.5

[0118] MDWI = (NIR - SWIR) / (NIR + SWIR)

[0119] GNDVI = (NIR - Green) / (NIR + Green)

[0120] RVI = NIR / Red

[0121] SIPI = (NIR - Red) / (NIR - Blue)

[0122] The Blue, Green, Red, NIR, and SWIR in the formula represent the blue band, green band, red band, near infrared band, and short-wave infrared band, respectively.

[0123] The above water index characteristics: Normalized Difference Water Index (NDWI), Modified Normalized Difference Water Index (MNDWI), Water Index (WI), the calculation formula is as follows:

[0124] NDWI = (Green - NIR) / (Green + NIR)

[0125] MNDWI = (Green - SWIR) / (Green + SWIR)

[0126] WI = Green / (NIR + SWIR)

[0127] Green, NIR, and SWIR in the formula represent green band, near infrared band, and short-wave infrared band, respectively.

[0128] The above texture features include: Angular Second Moment (ASM), Contrast, Correlation, Entropy, Homogeneity, Inverse Difference Moment (IDM), Difference Entropy. In the calculation of texture features, a 3*3 window is selected, and the results calculated in the four directions of 0°, 45°, 90°, and 135° are averaged. The calculation formulas for each texture feature are as follows:

[0129]

[0130] Where p(ij) is the element in the i-th row and j-th column of the gray-level co-occurrence matrix, and are the means of the rows and columns in the gray-level co-occurrence matrix, and are the standard deviations of the rows and columns in the gray-level co-occurrence matrix, respectively.

[0131] S144, after calculating the above features, first randomly obtain a number of points in each image label, and then if the number of label values ​​at the center point within a 5*5 range around the sampling point is greater than 80%, then retain this point as a training sample.

[0132] After obtaining the training samples, the spectral angle method is used to perform preliminary cleaning of the training samples. The calculation formula is:

[0133]

[0134] Where Pnorm is the normalized vector of the feature, Rnorm is the normalized vector of the reference feature, and is replaced by the average value of the feature vectors of each category. is the angle between the two vectors.

[0135] For each category of samples, the spectral angle between them and their respective reference features is calculated, and samples within the range of the mean ± 2 times the standard deviation are taken as subsequent training samples, and samples outside the range are discarded.

[0136] The training samples after preliminary cleaning are divided into training set and test set in a ratio of 7:3.

[0137] S145, considering that the label accuracy in the training data may be low, which will further affect the accuracy of the subsequent classification model, to address this problem, the samples are further denoised, which is divided into the following steps: (1) estimating the joint distribution of noise labels and true labels; (2) filtering noise samples; (3) training sample optimization model. The details are as follows:

[0138] The joint distribution calculation formula of the noise label and the true label is:

[0139]

[0140] in, For sample labels The number of is the number of samples in the classification confusion matrix that are actually ii and predicted to be jj, In the classification confusion matrix, the true value is ii and the prediction is The number of samples, m is the number of categories, Indicates accumulation, is the joint probability value calculated for each category.

[0141] Based on the joint distribution, a method is proposed to identify and filter noise samples. The method is to calculate the average joint probability value of each label category. When the predicted probability is less than this value, it is a noise sample and is removed.

[0142] After filtering the noise samples, the XGBoost classification algorithm is used to train each partition model. In the classification, the grid search algorithm is used to optimize the hyperparameters.

[0143] S146, classify and predict all images within the image range of each partition, and use the obtained results as prediction results. The calculation of the score is When the value is large, it is considered that the prediction result is only a correction of the original label result. When the value is small, it means that the predicted result and the original label result are very different, and it is impossible to judge which result is better, so it is discarded. The F1 calculation formula is as follows:

[0144]

[0145] in, is the accuracy, is the recall rate, To calculate the F1 score, the threshold is set within one standard deviation of the average value of the F1 score calculated for all prediction results and the original label in each partition, so as to retain the optimized sample data with high precision.

[0146] Step S2 specifically includes the following steps:

[0147] S21: putting the corrected and refined samples finally selected in step S1 into the sample database, and storing the early pre-trained model into the model database;

[0148] S22, if the optimized samples are directly used to continue training on the basis of the pre-trained model, there will be a problem of model "forgetting" to a certain extent, that is, the model forgets what it has learned before and starts training again, so incremental learning is needed to complete subsequent training. Incremental learning has certain requirements for samples, specifically: randomly select 25% of the screened optimized samples with high confidence in the sample database, replace these samples with the original labels to update the entire training set, so that there are a small number of original samples in the training set, and there are also optimized samples to solve the problem of decreased accuracy of the model due to the "forgetting" phenomenon.

[0149] In the S3 stage, the dataset selected in the S2 stage is used to continue training based on the pre-trained model, and the number of iterations is set. number , learning rate, activation function and other parameters.

[0150] In summary, the present invention calculates the LN score and RC score of the two quality assessment indexes through the weights of the pre-trained model trained earlier by the FCT lightweight semantic segmentation model, and uses the Beta mixture model to model the two quality assessment index score results, and divides all samples into two parts, clean samples and noise samples, at the image level; a label correction and refinement model is established on the clean samples, and all labels are optimized using the model, and then the refined labels and part of the original labels are used to continue training the incremental learning model. In addition, the present invention not only realizes the rapid training of the model in the presence of a large number of samples, but also further achieves the purpose of correcting and refining the samples and removing redundant samples, while reducing the consumption of training resources without reducing the accuracy of the model.

[0151] The present invention also discloses a computer-readable storage medium and a computer system, wherein a computer program is stored on the computer-readable storage medium, and after the computer program is run, the above method is executed. A computer system includes a processor and a storage medium, wherein the storage medium stores a computer program, and the processor reads and runs the computer program from the storage medium to execute the above method.

[0152] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0153] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0154] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0155] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, a server, or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0156] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but should be granted the widest scope consistent with the principles and novel features disclosed herein.

[0157] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight semantic segmentation model remote sensing sample optimization method, characterized in that: The following steps are involved: S1, collect publicly available remote sensing images and their corresponding classification labels on the Internet to form sample data for semantic segmentation, perform data enhancement on the samples, build a lightweight semantic segmentation FCT network model, build a Beta hybrid model, and establish a label correction and refinement model; Specifically, S11 collects remote sensing images and their corresponding classification labels to form sample data for semantic segmentation, and enters the data enhancement module. The data enhancement methods specifically include random rotation, random scaling, random cropping, horizontal flipping, random erasing, image mixing, and color jittering; S12, constructing a lightweight semantic segmentation FCT network model, and using the existing samples, i.e., the sample data collected in step S1, to pre-train the FCT network model; S13, for all samples, including images and remote sensing images in their corresponding labels, use the pre-trained model to output the probability map of the last layer of the network, and then calculate the LN score and RC score of the two quality assessment index scores respectively. The Beta mixture model is used to model the two quality assessment index score results, and all samples are divided into clean samples and noise samples at the image level; LN score is defined as the expected norm of the loss gradient at time t : ； Among them, x represents the image data in the sample, y represents the label data in the sample, and W t are the model parameters at time t, is the model's predicted probability for the kth category, K represents the number of categories, and y k is the one-hot encoding of the true label, is the logit gradient for the kth category, Indicates the calculation of the square of the L2 norm, E indicates the calculation of the mathematical expectation value, Indicates accumulation, Indicates the calculated LN score value at time t; The RC score is defined as the expected value of the square of the L2 norm between the predicted probability vector and the true label vector; for a training sample (x, y), its RC score is expressed as: ； Among them, x represents the image data in the sample, y represents the label data in the sample, and W t are the model parameters at time t, is the model's predicted output for the input x image data, is the one-hot encoding of the true label, Represents the square of the L2 norm, which calculates the square error between the predicted output and the true label. Represents the mathematical expectation of the L2 norm square of the predicted label and the true label at time t, Indicates the calculated RC score value at time t; S14, based on the method of step S13, select clean samples to establish a label correction refinement model and optimize the samples; S2, putting the samples selected in step S1 after correction and refinement into the sample database, and storing the pre-trained model trained earlier, i.e., the model obtained by pre-training the FCT network model in step S12, into the model database, and further screening and processing the samples according to the requirements of the incremental learning technology for the samples; S3, using the data set screened in step S2 to perform further training based on the pre-trained model.

2. The method according to claim 1, characterized in that In step S12, a lightweight semantic segmentation FCT network model is constructed, which specifically includes: an encoder module, a decoder module, an efficient transformation network module, a feature enhancement module, and a pixel attention mechanism module; The encoder module includes a lightweight expansion bottleneck layer module, and the structure of the lightweight expansion bottleneck layer module consists of 1*1 convolution, 1*3 convolution and channel attention; The efficient transformation network module uses the self-attention mechanism module as the capsule network, and its formula is as follows: ； in, , , denote the query vector, key vector and value vector respectively, represents the transpose of the key vector; D is the embedding dimension of the input feature layer; is the activation function, O is the normalized matrix; Pixel attention mechanism module: The pixel attention mechanism learns weights based on the importance of features at different pixel positions. The formula is: ； Where X is the output feature of the last dilated bottleneck layer in the decoder module, represents the sigmoid function, Represents the convolution layer, which means the convolution operation with a convolution kernel of 1*1. Represents the output features after the pixel attention mechanism operation; Feature enhancement module: modeling feature dependencies from the channel dimension and spatial dimension respectively.

3. The method according to claim 1, characterized in that The step S14 of establishing a label correction refinement model and optimizing the sample specifically includes the following steps: S141, in the clean sample and noise sample data sets, respectively calculate the distance weight within the range of 100 samples around each sample, the weight selects the simplified Gaussian function, and calculates the purity of each sample in the entire spatial range according to the distance weight; S142, calculating the coordinates of the four corner points of the clean sample and the purity of the sample, and clustering them using Hierarchical clustering; S143, for the image data corresponding to the samples in each cluster after clustering, calculate the building index feature, vegetation index feature, water index feature and grayscale symbiotic texture matrix feature; S144, after calculating the above features, first randomly obtain a number of points in each image label, and then if the number of label values ​​at the center point within a 5*5 range around the sampling point is greater than 80%, then retain this point as a training sample; S145, in the training samples, further denoising the samples at the pixel level according to the process of estimating the joint distribution of the noise labels and the true labels, filtering the noise samples, and optimizing the training sample model; S146, using the model optimized by the trained samples to predict all samples, calculating the F1 score index for the optimized results and the original labels, performing a histogram analysis on the F1 score values, setting a threshold and selecting samples that meet the threshold conditions as high-quality samples.

4. The method according to claim 1, characterized in that Step S2 specifically includes the following steps: S21, putting the sample selected in step S1 after correction and refinement into the sample database, and storing the pre-trained model into the model database; S22, randomly select 25% of the screened optimized samples with high confidence in the sample database, and replace these samples with the original labels to update the entire training set.

5. The method according to claim 1, characterized in that: In step S3, the data set selected in step S2 is used to perform further training on the basis of the pre-trained model, and parameters are set, including the number of iterations, learning rate, and activation function.

6. A computer-readable storage medium, characterized in that: A computer program is stored on the medium, and after the computer program is run, the method according to any one of claims 1 to 5 is executed.

7. A computer system, characterized in that: The method comprises a processor and a storage medium, wherein a computer program is stored in the storage medium, and the processor reads and runs the computer program from the storage medium to execute the method as claimed in any one of claims 1 to 5.

Images