A remote sensing image enhancement classification method based on object association features

Abstract

The present invention relates to the field of remote sensing technology and artificial intelligence pattern recognition, and specifically to a remote sensing image enhancement classification method based on object association features. Acquire multiple sample data in a database; extract the features of each sample and combine them in pairs to construct a binary training set; input the binary training set into a graph convolutional network model for training; extract the features of any two pixels in the target remote sensing image and combine them in pairs to construct a target binary set; input the target binary set into a classification model to obtain the classification result of the target binary; decompose the target binary to obtain the category of each pixel; repeatedly obtain multiple classification results for each pixel; and use the majority voting method to obtain the final classification result of each pixel to complete the classification of the target remote sensing image. The present invention utilizes the feature associations between objects to combine into binary, and can output classification results with diversity, thereby ensuring the rapidity and stability of classification.

Description

Technical Field

[0001] The present invention relates to the fields of remote sensing technology and artificial intelligence pattern recognition, and particularly relates to a remote sensing image enhancement classification method based on object association features. Background Art

[0002] Machine learning provides the most direct and effective means for the automatic classification of remote sensing images and is widely used in the production of products such as land use. For a long time, researchers have been committed to developing and applying new machine learning algorithms to improve the accuracy and efficiency of the automatic classification of remote sensing images. Some typical classification algorithms, such as the maximum likelihood method, minimum distance method, support vector machine, naive Bayes, and artificial neural network, etc., have played a very important role in the automatic classification of images. However, these traditional algorithms can only form a single classifier and have disadvantages such as insufficient representation ability, difficulty in distinguishing similar objects, and low accuracy.

[0003] Ensemble learning effectively solves the deficiencies of single classifiers. By using different feature spaces, training sets, or classification algorithms to train multiple classifiers, and then through a certain ensemble strategy, the outputs of all classifiers are integrated to obtain better results than a single classifier. By improving the diversity of the outputs of the base classifiers, ensemble learning can greatly expand the improvement space of classification accuracy. Due to its advantages, the idea of ensemble learning has once become a research hotspot in the fields of machine learning and remote sensing image classification. Some typical methods, such as AdaBoost, random forest, XGBoost, etc., have been widely studied and applied and achieved excellent results.

[0004] For ensemble learning, the diversity of the base classifiers is the basic premise of ensemble learning because only when there are differences between the base classifiers can there be a complementary relationship between them. The ensemble strategy is the key to ensemble learning because only a good ensemble strategy can make the advantages of different base classifiers complementary, thereby improving the classification accuracy. Therefore, typical ensemble learning is generally divided into three steps. First, a series of basic classifiers are trained using samples; second, the basic classifiers are used to classify the input instances and generate a series of different classification results; finally, a certain ensemble strategy is adopted to integrate all the classification results to obtain more accurate results.

[0005] However, this typical ensemble learning method has certain drawbacks. The most important one is that training a large number of base classifiers is required to implement each ensemble learning classification framework, and the diversity of these classifiers often needs to be obtained through different training sets, different feature spaces, and classification algorithms, which undoubtedly increases the complexity of the algorithm and the running efficiency of the model. Especially when dealing with some time-consuming base classification algorithms (such as deep neural networks), the efficiency of ensemble learning will be greatly reduced, hindering the common improvement of learning accuracy and efficiency. Summary of the Invention

[0006] To solve the problems existing in the prior art, the present invention provides a remote sensing image enhancement classification method based on object association features. The solution includes: obtaining a plurality of sample data in a database; extracting the features of each sample and combining them pairwise to construct a binary training set; inputting the binary training set into a graph convolutional network model for training; extracting the features of any two pixels in the target remote sensing image and combining them pairwise to construct a target binary set; inputting the target binary set into a classification model to obtain the classification results of the target binary; decomposing the target binary to obtain the category of each pixel; repeating to obtain multiple classification results of each pixel; and using the majority voting method to obtain the final classification result of each pixel to complete the classification of the target remote sensing image. The present invention combines the feature associations between objects into binaries, which can output diverse classification results and ensure the rapidity and stability of classification.

[0007] The present invention adopts the following technical solution. A remote sensing image enhancement classification method based on object association features includes:

[0008] S1. Obtain a plurality of sample data in a database and construct a sample set according to the feature values and feature types of the sample data;

[0009] S2. Extract the features of each sample in the sample set and combine them pairwise to construct a binary training set;

[0010] S3. Input the binary training set into a graph convolutional neural network model for training to obtain a trained classification model;

[0011] S4. Extract the features of any two pixels in the target remote sensing image in a random and non-replacement manner and combine them pairwise to construct a target binary set;

[0012] S5. Input the target binary set into the trained classification model to obtain the classification results of each target binary;

[0013] S6. Decompose the target binary based on the classification results of each target binary to obtain the category of each pixel in the target binary;

[0014] S7. Repeat steps S4 - S6 to obtain multiple classification results of each pixel in the target remote sensing image;

[0015] S8. Use the majority voting method to obtain the final classification result of each pixel in the target remote sensing image from the multiple classification results to complete the classification of the target remote sensing image.

[0016] Further, constructing a sample set according to the feature values and feature categories in the sample remote sensing image includes:

[0017] Taking the feature type included in each sample data as the x-axis and the feature value as the y-axis, a Cartesian plane coordinate system is established to obtain the feature curve of each sample data;

[0018] Taking the position of each feature value in the feature curve as a node, an undirected complete graph feature of the sample data is constructed;

[0019] A sample set is constructed according to the undirected complete graph features corresponding to all sample data.

[0020] Furthermore, the features of each sample in the sample set are extracted and combined pairwise, including:

[0021] Arbitrarily select two samples s i =(g i , h i ) and sample s j =(g j , h j ) from the sample set, where g i is the undirected complete graph feature of the i-th sample data, h i is the class label of the i-th sample data, g j represents the undirected complete graph feature of the j-th sample data, and h j represents the class label of the j-th sample data;

[0022] The two selected samples are combined to obtain a binary pair s ij =(g ij , h ij ), where g ij =g i ∪g j .

[0023] Furthermore, based on the classification results of each target binary pair, the target binary pair is decomposed to obtain the class of each pixel in the target binary pair, including:

[0024] Input the target feature binary pair p ab into the trained classification model for classification to obtain the classification result l;

[0025] After decomposing l, the class labels c a and c b of p a and p b are obtained respectively. Specifically:

[0026]

[0027] c b =l (k-1)％m+1

[0028] Among them, % represents the modulo operation, k represents the category serial number, and m represents the total number of categories.

[0029] Further, the majority voting method is used to obtain the final classification result of each pixel in the target remote sensing image from the multiple classification results, including:

[0030] Convert the multiple classification results of each pixel into one-hot encoding and perform bitwise addition to obtain the category vector corresponding to each pixel;

[0031] Use a mapping function to calculate the category vector to obtain the final classification result of each pixel.

[0032] The beneficial effects of the present invention are as follows: The present invention combines the samples in the training set in pairs and sets labels for each category combination to obtain a series of binary pairs. Then, by calculating the graph features of each binary pair, a training set based on binary pairs is obtained to train the graph convolutional neural network classifier, which can fully learn the correlation information between features, obtain more abstract common features therefrom, and thus improve the separability between different categories; Randomly obtain two pixels from the remote sensing image to form a binary pair, and predict the label of each binary pair through the classifier, which has a broader expansion space compared to single-pixel classification; Use two-dimensional matrix indexing to split the binary pair to obtain the category corresponding to the input entity. After repeating multiple times, the multi-classification result of each pixel on the image is finally obtained, and then the majority voting method is used for integration to obtain a more accurate output, thereby realizing the integrated learning remote sensing image classification based on correlation features, achieving the diversity of classification results without training multiple classification models, and effectively improving the classification accuracy through integration, ensuring the rapidity, stability, and generality of the classification model. Description of the Drawings

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0034] Figure 1 It is a schematic flowchart of a remote sensing image enhancement classification method based on object correlation features according to an embodiment of the present invention;

[0035] Figure 2 It is a schematic diagram of the principle of constructing graph features according to an embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram of the relationship index between the binary pair label L and the category label C according to an embodiment of the present invention. Detailed implementation manners

[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0038] A schematic flowchart of a remote sensing image enhancement classification method based on object association features in an embodiment of the present invention is as Figure 1 shown, including:

[0039] S1. Obtain multiple sample data in the database, and construct a sample set according to the feature values and feature types of the sample data;

[0040] In the embodiment of the present invention, a sample set S = {F, H} = {(f 1 , h 1 ), (f 2 , h 2 ), (f 3 , h 3 ),...,(f n , h n )}(f j ∈F, h j ∈H, j = (1, 2, 3,..., n)) is established based on the sample data extracted from the database, where f j ∈F, h j ∈H, j = (1, 2, 3,..., n)), n is the total number of samples, F and H respectively represent the feature space and the class label space. Let C = {C 1 , C 2 , C 3 ,..., C m} be the class label set, m is the total number of classes, and H ∈ C; f j = (f j1 , f j2 ,..., f jo ) is the feature of the j-th sample, and o is the number of features;

[0041] For the samples in S, construct the graph feature of each sample to obtain a sample set S' = {G, H} based on the graph feature, where G = {G1, G2,..., Gn} is a graph feature set. As Figure 2 shown, a schematic diagram of the construction principle of the graph feature in the embodiment of the present invention is given, and the specific steps are as follows:

[0042] For the feature f j = (f j1, f j2 ,..., f jo ), with the feature index as the x-axis and the feature value as the y-axis, establish a Cartesian plane coordinate system to obtain the feature curve t of sample j j ;

[0043] Take the positions of different feature values on curve t j as nodes and the connections between nodes as edges to establish an undirected complete graph g j =(v j , e j ); where v j is the set of nodes. In addition to storing its own attributes, each node also takes the position in polar coordinates and plane coordinates as attributes; e j is the set of edges. Each edge stores the relationship values under different parameter conditions between nodes. In the embodiments of the present invention, it includes the azimuth vector, mean value, and distance between two nodes. Perform the above operations on all samples in S, and finally obtain the training set S' = {G, H} based on graph features.

[0044] In the embodiments of the present invention, the classification objects include, in addition to remote sensing images, other data described by one-dimensional arrays to describe features; specifically, the objects to be classified can come from any type of remote sensing images, other types of multi-band remote sensing images, and any other high-dimensional data sets described by one-dimensional arrays to describe different features. The embodiments of the present invention do not make unique limitations.

[0045] Of course, in this embodiment, the objects targeted are not limited to remote sensing images only, but can also be other natural images, and any other data sets described by one-dimensional arrays to describe different features. In addition, the construction of graph features of samples or pixels can use not only logical operations of graphs, but also various functions, as well as other transformation methods, etc.

[0046] S2. Extract the features of each sample in the sample set and combine them pairwise to construct a binary training set;

[0047] Taking the pairwise combination of elements in C as an example, the set D can be expressed as:

[0048]

[0049] Each element in set D represents a binary label, while C i C j represents the label index corresponding to the binary formed by samples with class labels C i and C j respectively.

[0050] Sort the elements in D row by row to further obtain the binary class label set L = {L 1 , L2 , L 3 ,..., L k ,..., L m×m}(k ∈ [1, m×m]).

[0051] Then, in the embodiment of the present invention, a sample s' is randomly obtained from the training set S' = {G, H} based on graph features i =(g i , h i ) and sample s' j =(g j , h j ) (where h i = C i , h j = C j ), and the binary formed by their combination is s' ij =(g ij , h ij )=(g ij , l i×m+j ), where g ij is the graph feature, g ij = g i ∪ g j , h ij is the binary label index corresponding to h i and h j . After sorting by rows in D, the obtained binary class label is l i×m+j ∈ L; all samples in S' are combined in pairs, and finally the feature binary training set S doublets is obtained.

[0052] S3. Input the feature binary into the graph convolutional network model for training to obtain a trained classification model;

[0053] That is, in the embodiment of the present invention, S doublets is used as the training set, and the graph convolutional neural network is used to train the classification model M;

[0054] It should be noted that in the embodiment of the present invention, the method for constructing the classifier includes the graph convolutional neural network and the variant algorithms of the graph convolutional neural network. The specific method adopted is not uniquely limited in the embodiment of the present invention.

[0055] S4. Extract the features of any two pixels in the target remote sensing image in a random and non-replacement manner for pairwise combination to construct a target binary set;

[0056] For any two pixels P a and P b in the target remote sensing image R, the features of P a and P b are respectively obtainedGraph feature g a =(v a , e a ), g b =(v a , e b ); Further use the method in step S2 to obtain P a and P b of the binary body P ab graph feature g ab = g a ∪ g b ; Input P ab into the classification model M to obtain the classification result l (l ∈ L) of this binary body.

[0057] S5. Input the set of target feature binary bodies into the trained classification model to obtain the classification results of each target feature binary body;

[0058] In the embodiment of the present invention, pixels are obtained from R by a random sampling without replacement method to construct binary bodies, and the class labels of the pixels are obtained through the trained classification model M. All pixels in the target remote sensing image are traversed and combined until the class labels of all binary bodies corresponding to the pixels are obtained;

[0059] S6. Decompose the target binary body based on the classification result of each target binary body to obtain the class of each pixel in the target binary body;

[0060] Finally, decompose l to obtain the predicted class labels c a and c b (c a ∈ C, c b ∈ C); As a shown is the relationship between the class label C and the binary body label L; The calculation method for decomposing l to obtain p b and p a is as follows: b Figure 3

[0061]

[0062] c b = l (k-1)％m+1

[0063] where % represents the modulo operation, k represents the class serial number, and m represents the total number of classes.

[0064]

[0065] S7. Repeat steps S4 - S6 to obtain multiple classification results of each pixel in the target remote sensing image;

[0065] In the embodiment of the present invention, the technical content of steps S4 - S6 is repeated w times to obtain the multi - layer classification result Q' = {Q 1 , Q 2 ,..., Q w} of the target remote - sensing image.

[0066] S8. Use the majority voting method to obtain the final classification result of each pixel in the target remote - sensing image from the multiple classification results, and complete the classification of the target remote - sensing image.

[0067] Taking the class label c i as an example, its one - hot encoding can be represented as a vector e=(e 1 , e 2 , e 3 ,..., e m ), where e i is 1 and the other elements are all 0. Convert the class label {q 1 , q 2 , q 3 ,..., q w} of a pixel in Q' into one - hot encoding {v 1 , v 2 , v 3 ,..., v w} and perform bit - by - bit addition according to the following formula to obtain a feature vector v:

[0068]

[0069] After that, for v, use the following formula to obtain the final classification result:

[0070] c = argmax(v)

[0071] Finally, perform the same operation on all pixels in the multi - layer classification result Q' of the target remote - sensing image to obtain the final classification result of the image R.

[0072] Specifically, as shown in Figure 1 , first, randomly select target pixels A, B, C, D, E from the temporal image plane; then pair them up two by two to get the dyads AB, AC, AD, AE, BC, BD, BE, CD, CE, DE; then use the above method to classify these dyads to obtain the diversified classification results of each pixel. In this way, processing all the remaining pixels will obtain the multi - classification results of the remote - sensing image. Finally, through the voting method for these classification results, a more accurate classification output is obtained.

[0073] In summary, taking remote sensing images as an example, the present invention combines samples in the training set in pairs and sets labels for each sample combination to obtain a series of binary tuples; then, calculates the graph features of each binary tuple to obtain a training set based on binary tuples for training a graph convolutional neural network classifier; thereafter, randomly obtains two pixels from the remote sensing image to form a binary tuple, and predicts the label of each binary tuple through the classifier; thereafter, uses two-dimensional matrix indexing to split the binary tuple to obtain the category corresponding to the input pixel; repeats this process multiple times, and finally obtains the multi-classification result of each pixel on the image; thereafter, uses the method of majority voting for integration to obtain a more accurate output; realizes the integrated learning remote sensing image classification based on associated features; the present invention realizes the diversity of classification results without training multiple classification models, and effectively improves the classification accuracy through integration, ensuring the stability and generality of the classification model.

[0074] Compared with the integrated classification based on multiple models, the present invention only needs to train one model to achieve integrated classification, greatly shortening the time required for training the integrated classification framework, and having a greater improvement in efficiency; in addition, converting the original features of the data into graph features can fully learn the associated information between features, obtain more abstract common features therefrom, and thus improve the separability between different categories; in addition, constructing objects into binary tuples for classification has a broader expansion space compared with single-pixel classification. On this basis, multiple objects can be constructed into a combined body for classification, and it is expected to make further breakthroughs in the construction of an integrated classification framework without multiple classifiers.

[0075] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A remote sensing image enhancement classification method based on object association features, characterized in that: include: S1. Obtain multiple sample data in the database, and construct a sample set according to the feature values ​​and feature types of the sample data; S2. Extract the features and category labels of each sample in the sample set and combine them in pairs to construct a binary training set; including: Randomly select two samples s from the sample set i =(g i ,h i ) and sample s j =(g j ,h j ), where g i is the undirected complete graph feature of the i-th sample data, h i is the category label of the i-th sample data, g j represents the undirected complete graph feature of the jth sample data, h j Represents the category label of the jth sample data; combine the two selected samples to obtain the binary s ij =(g ij ,h ij ), where g ij =g i ∪g j ; S3. Inputting the binary training set into the graph convolutional neural network model for training to obtain a trained classification model; S4. extracting features of any two pixels in the target remote sensing image by random without replacement, combining them in pairs, and constructing a target binary set; S5. Input the target binary set into the trained classification model to obtain the classification result of each target binary; S6. Decomposing the target binary based on the classification result of each target binary to obtain the category of each pixel in the target binary; including: The target feature binary p ab Input the trained classification model for classification and obtain the classification result l; After decomposing l, we can get p a and p b The category label c a and c b , specifically: c b ＝l (k-1)％m+1 Among them, % represents the modulo operation, k represents the category number, and m represents the total number of categories; S7. Repeat steps S4-S6 to obtain multiple classification results for each pixel in the target remote sensing image; S8. Using the majority voting method to obtain the final classification result of each pixel in the target remote sensing image from the multiple classification results, complete the classification of the target remote sensing image.

2. The remote sensing image enhancement classification method based on object association features according to claim 1 is characterized in that: The sample set is constructed based on the feature values ​​and feature types in the sample remote sensing images, including: A Cartesian plane coordinate system is established with the feature type contained in each sample data as the x-axis and the feature value as the y-axis to obtain the feature curve of each sample data; Taking the position of each eigenvalue in the characteristic curve as a node, constructing an undirected complete graph feature of the sample data; Construct a sample set based on the undirected complete graph features corresponding to all sample data.

3. The remote sensing image enhancement classification method based on object association features according to claim 1, characterized in that: The final classification result of each pixel in the target remote sensing image is obtained from the multiple classification results using the majority voting method, including: Convert the multiple classification results of each pixel into one-hot encoding and add them bit by bit to obtain the category vector corresponding to each pixel; The category vector is calculated using a mapping function to obtain a final classification result for each pixel.

Images