A remote sensing image classification method, device, equipment and storage medium
By constructing solar light incident angle images and convolutional neural networks, a remote sensing image classification model that takes into account sunlight is generated, which solves the problem of limited land object recognition accuracy in remote sensing images and achieves higher classification accuracy.
Patent Information
- Application Number
- CN202111227079.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-21
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-10-21
AI Technical Summary
The existing high-resolution remote sensing image classification method fails to effectively consider the impact of the direction of the sun's incident light on the spectral characteristics, resulting in limited accuracy of land objects recognition, and there is a problem of "large intra-class variance and small inter-class variance".
By constructing solar light incident angle images, combining the three-dimensional surface model of the ground surface and convolutional neural network, a remote sensing image classification model that takes into account the sunlight is generated, and participating in the classification process of remote sensing images is carried out to improve the accuracy of ground object recognition.
It effectively improves the accuracy of land objects recognition, solves the problem of insufficient recognition ability caused by spectral feature differences, and enhances classification accuracy.
Smart Images

Figure CN113989664B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image classification method, device, equipment and storage medium. Background Art
[0002] Spectral features are the most commonly used remote sensing image classification features, which are used to reflect the unique electromagnetic wave reflection characteristics of the ground objects. The pixel value of the remote sensing image is not only related to the reflection characteristics of the ground objects, but also to the direction of the incident light. The angle between the normal line of the ground object surface and the direction of the incident light affects the direct energy of sunlight received by the ground object, which in turn affects the spectral value of the ground object in the remote sensing image, resulting in the phenomenon of "different spectra for the same object and the same spectrum for different objects", such as the sun-facing side of a triangular roof is obviously brighter than the sun-facing side.
[0003] The existing high-resolution remote sensing image classification methods do not consider the impact of the direction of the incident sunlight on the spectrum, resulting in the problem of "larger variance within the class and smaller variance between classes" in the image features used to identify objects. Therefore, the mapping relationship between image features and object targets in the classifier is very complex, which restricts the improvement of classification accuracy. There is currently no effective solution. Summary of the invention
[0004] In view of this, embodiments of the present invention are intended to provide a remote sensing image classification method, apparatus, device and storage medium.
[0005] The technical embodiment of the embodiment of the present invention is implemented as follows:
[0006] An embodiment of the present invention provides a remote sensing image classification method, the method comprising:
[0007] Obtaining a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image;
[0008] Slicing the remote sensing image, the sunlight incident angle image, and the annotated image to obtain a slice image;
[0009] Determining a remote sensing image classification model that takes sunlight into account according to the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image;
[0010] The ground objects in the remote sensing image are classified based on the remote sensing image classification model taking sunlight into consideration.
[0011] In the above solution, the step of obtaining the sunlight incident angle image of the first object in the remote sensing image includes:
[0012] Determining normal parameters corresponding to the first ground object in the remote sensing image according to the three-dimensional surface model of the ground;
[0013] Obtaining shooting parameters of the remote sensing image, and determining position parameters of the sun according to the shooting parameters;
[0014] The sunlight incident angle image is determined based on the normal parameters corresponding to the first ground object and the position parameters of the sun.
[0015] In the above scheme, the position parameters of the sun include at least the altitude parameters of the sun and the azimuth parameters of the sun; and determining the sunlight incident angle image based on the normal parameters corresponding to the first ground object and the position parameters of the sun includes:
[0016] Determine a first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first ground object;
[0017] Determine a second direction vector of the incident light from the sun on the first ground object in the preset coordinate system according to the position parameter of the sun;
[0018] The sunlight incident angle image is determined based on the first direction vector and the second direction vector.
[0019] In the above solution, the slicing process of the remote sensing image, the sunlight incident angle image and the annotated image to obtain the sliced image includes:
[0020] Obtaining a first slice image of a first preset size according to the remote sensing image, the sunlight incident angle image and the annotated image;
[0021] The first slice image is cut into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
[0022] In the above scheme, determining the remote sensing image classification model taking sunlight into account according to the slice image and the preset convolutional neural network model includes:
[0023] Obtaining a training data set according to the slice image;
[0024] The training data set is input into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account.
[0025] In the above scheme, the training data set includes spectral images of a first preset threshold number of bands in the remote sensing image, the sunlight incident angle images of a second preset threshold number of bands, and the annotated images, and the first preset threshold is greater than the second preset threshold; the step of inputting the training data set into the preset convolutional neural network model to determine the remote sensing image classification model taking sunlight into account includes:
[0026] Inputting the spectral image of the first preset threshold wavebands and the sunlight incident angle image of the second preset threshold wavebands into the preset convolutional neural network model to obtain an output result;
[0027] Determining a loss parameter of the preset convolutional neural network model according to the output result and the annotated image;
[0028] Determining a correction value of each layer parameter in the preset convolutional neural network model based on the loss parameter;
[0029] The preset convolutional neural network model is optimized according to the correction value until the loss parameter of the optimized preset convolutional neural network model is less than a third preset threshold, thereby obtaining the remote sensing image classification model that takes sunlight into account.
[0030] In the above solution, the method further includes:
[0031] Obtaining a verification data set according to the slice images;
[0032] The accuracy of the remote sensing image classification model taking sunlight into account is evaluated based on the verification data set.
[0033] In the above scheme, the accuracy evaluation of the remote sensing image classification model taking sunlight into account based on the verification data set includes:
[0034] Inputting the verification data set into the remote sensing image classification model taking sunlight into account to obtain a classification result;
[0035] Determine the target classification result corresponding to the pixel with the highest overlap in the classification result;
[0036] The accuracy of the remote sensing image classification model taking sunlight into account is evaluated according to the target classification result.
[0037] An embodiment of the present invention provides a remote sensing image classification device, the device comprising: an acquisition unit, a processing unit, a determination unit and a classification unit, wherein:
[0038] The obtaining unit is used to obtain a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image;
[0039] The processing unit is used to perform slice processing on the remote sensing image, the sunlight incident angle image, and the annotated image to obtain a slice image;
[0040] The determination unit is used to determine a remote sensing image classification model that takes sunlight into consideration according to the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image;
[0041] The classification unit is used to classify the ground objects in the remote sensing image based on the remote sensing image classification model taking sunlight into consideration.
[0042] In the above scheme, the acquisition unit is also used to determine the normal parameters corresponding to the first ground object in the remote sensing image based on the three-dimensional surface model of the earth; obtain the shooting parameters of the remote sensing image, and determine the position parameters of the sun based on the shooting parameters; and determine the sunlight incident angle image based on the normal parameters corresponding to the first ground object and the position parameters of the sun.
[0043] In the above scheme, the position parameters of the sun include at least the altitude parameters and the azimuth parameters of the sun; the acquisition unit is also used to determine the first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first object; determine the second direction vector of the incident light of the sun shining on the first object in the preset coordinate system according to the position parameters of the sun; and determine the sunlight incident angle image based on the first direction vector and the second direction vector.
[0044] In the above scheme, the processing unit is also used to obtain a first slice image of a first preset size based on the remote sensing image, the sunlight incident angle image and the annotated image; cut the first slice image into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
[0045] In the above scheme, the determination unit is also used to obtain a training data set based on the slice image; and input the training data set into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account.
[0046] In the above scheme, the training data set includes spectral images of a first preset threshold band, the sunlight incident angle images of a second preset threshold band and the annotated images in the remote sensing image, and the first preset threshold is greater than the second preset threshold; the determination unit is also used to input the spectral images of the first preset threshold band and the sunlight incident angle images of the second preset threshold band into the preset convolutional neural network model to obtain an output result; determine the loss parameters of the preset convolutional neural network model according to the output result and the annotated image; determine the correction values of the parameters of each layer in the preset convolutional neural network model based on the loss parameters; optimize the preset convolutional neural network model according to the correction values until the loss parameters of the optimized preset convolutional neural network model are less than the third preset threshold, so as to obtain the remote sensing image classification model that takes sunlight into account.
[0047] In the above solution, the device further comprises an evaluation unit,
[0048] The obtaining unit is further used to obtain a verification data set according to the slice image;
[0049] The evaluation unit is used to evaluate the accuracy of the remote sensing image classification model taking sunlight into account based on the verification data set.
[0050] In the above scheme, the evaluation unit is also used to input the verification data set into the remote sensing image classification model that takes sunlight into account to obtain a classification result; determine the target classification result corresponding to the pixel with the highest overlap in the classification result; and perform accuracy evaluation on the remote sensing image classification model that takes sunlight into account based on the target classification result.
[0051] An embodiment of the present invention provides a remote sensing image classification device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, any step of the above-mentioned method is implemented.
[0052] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any step of the above-mentioned method is implemented.
[0053] The embodiments of the present invention provide a remote sensing image classification method, device, equipment and storage medium, which include: obtaining a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image; slicing the remote sensing image, the sunlight incident angle image and the labeled image to obtain a slice image; determining a remote sensing image classification model that takes sunlight into account based on the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image; and classifying the objects in the remote sensing image based on the remote sensing image classification model that takes sunlight into account. The technical solution of the embodiments of the present invention is adopted to effectively improve the accuracy of object recognition by involving the sunlight incident angle image in remote sensing image classification. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 A schematic diagram of a remote sensing image classification method implementation flow in an embodiment of the present invention;
[0055] Figure 2 A schematic diagram of a process of constructing an incident angle image in a remote sensing image classification method according to an embodiment of the present invention;
[0056] Figure 3 A schematic diagram of another implementation flow of a remote sensing image classification method according to an embodiment of the present invention;
[0057] Figure 4 A schematic diagram of the structure of a remote sensing image classification device according to an embodiment of the present invention;
[0058] Figure 5 The figure is a schematic diagram of a hardware entity structure of a remote sensing image classification device in an embodiment of the present invention. DETAILED DESCRIPTION
[0059] To make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the specific technical solution of the invention will be further described in detail below in conjunction with the drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0060] This embodiment proposes a remote sensing image classification method, which is applied to design equipment. The functions implemented by the method can be implemented by calling program codes through a processor in the design equipment. Of course, the program codes can be stored in a computer storage medium. It can be seen that the computing device at least includes a processor and a storage medium.
[0061] This embodiment proposes a remote sensing image classification method. Figure 1 FIG. 1 is a schematic diagram of a remote sensing image classification method according to an embodiment of the present invention. Figure 1 As shown, the method includes:
[0062] Step S101: obtaining a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image.
[0063] In this embodiment, a remote sensing image classification method can be a high-resolution remote sensing image classification method; specifically, it can be a high-resolution remote sensing image classification method that takes into account the direction of incident sunlight in a three-dimensional scene; in practical applications, the remote sensing image classification method can also be called a land feature classification method that takes into account the direction of incident sunlight.
[0064] Obtaining the annotated image of the first object in the remote sensing image can be to manually annotate different types of object patches in a portion of the high-resolution remote sensing image, and generate an annotated image consistent with the spatial relationship of the original image pixels, wherein the pixel values on the annotated image represent the object category.
[0065] Obtaining the sunlight incident angle image of the first object in the remote sensing image can be as follows: determining normal parameters corresponding to the first object in the remote sensing image based on a three-dimensional surface model of the earth's surface; obtaining shooting parameters of the remote sensing image, and determining position parameters of the sun based on the shooting parameters; and determining the sunlight incident angle image based on the normal parameters corresponding to the first object and the position parameters of the sun.
[0066] Step S102: Slice the remote sensing image, the sunlight incident angle image, and the annotated image to obtain a slice image.
[0067] In this embodiment, the remote sensing image, the sunlight incident angle image, and the annotated image are sliced to obtain a slice image, which can be a first slice image of a first preset size obtained based on the remote sensing image, the sunlight incident angle image, and the annotated image; the first slice image is cut into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
[0068] Step S103: Determine a remote sensing image classification model that takes sunlight into account based on the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image.
[0069] In this embodiment, the preset remote sensing image may be a large number of remote sensing images stored in a database; the preset sunlight incident angle image may be a large number of sunlight incident angle images stored in a database; the preset remote sensing image and the preset sunlight incident angle image may be referred to as sample data;
[0070] The preset convolutional neural network model is determined by the preset remote sensing image and the preset sunlight incident angle image. The preset remote sensing image and the preset sunlight incident angle image can be subjected to deep learning network learning to obtain the mathematical relationship between the preset remote sensing image and the preset sunlight incident angle image, and the mathematical relationship is modeled using a convolutional neural network to obtain the preset convolutional neural network model.
[0071] As an example, the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image, and can use a deep learning method to mine the mathematical relationship between them from a large amount of sample data, and model the relationship using a convolutional neural network.
[0072] In practical applications, the deep learning network can be DeepLabv3. The present invention uses the DeepLabv3+ network as the basic network framework for remote sensing image semantic segmentation. The difference is that the DeepLabv3+ network inputs 3-band visible light images and does not consider the incident angle image. To address this problem, the present invention modifies the DeepLabv3+ network input layer to a 4-band input, where 3 bands are the 3 spectral bands of the remote sensing image and 1 band is the incident angle image.
[0073] Step S104: classifying the ground objects in the remote sensing image based on the remote sensing image classification model taking sunlight into consideration.
[0074] In this embodiment, classifying the ground objects in the remote sensing image based on the remote sensing image classification model taking sunlight into consideration may include inputting the remote sensing image into the remote sensing image classification model taking sunlight into consideration, and outputting a classification result of the ground objects in the remote sensing image.
[0075] The remote sensing image classification method provided by the embodiment of the present invention obtains a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image; the remote sensing image, the sunlight incident angle image, and the labeled image are sliced to obtain a slice image; a remote sensing image classification model taking sunlight into account is determined based on the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image; and the objects in the remote sensing image are classified based on the remote sensing image classification model taking sunlight into account. That is, by involving the sunlight incident angle image in the remote sensing image classification, the accuracy of object recognition is effectively improved.
[0076] In an optional embodiment of the present invention, obtaining the sunlight incident angle image of the first object in the remote sensing image includes: determining normal parameters corresponding to the first object in the remote sensing image based on a three-dimensional surface model of the earth's surface; obtaining shooting parameters of the remote sensing image, and determining position parameters of the sun based on the shooting parameters; and determining the sunlight incident angle image based on the normal parameters corresponding to the first object and the position parameters of the sun.
[0077] In this embodiment, determining the normal parameters corresponding to the first ground object in the remote sensing image according to the three-dimensional surface model of the ground surface may be by calculating the normal parameters corresponding to the first ground object point in the remote sensing image according to the three-dimensional surface model of the ground surface; wherein the first ground object point is any one of each ground object point in the remote sensing image; and the normal parameters may include a normal direction. As an example, calculating the normal parameters corresponding to the first ground object point in the remote sensing image according to the three-dimensional surface model of the ground surface may be by calculating the normal direction corresponding to a ground object point in the remote sensing image according to the three-dimensional surface model of the ground surface. In practical applications, the three-dimensional surface model of the ground surface may also be called a digital surface model, and calculating the normal direction corresponding to a ground object point in the remote sensing image according to the three-dimensional surface model of the ground surface may also be by calculating the normal direction of the ground object point corresponding to the pixel on the remote sensing image through the digital surface model.
[0078] The shooting parameters of the remote sensing image are obtained; wherein the shooting parameters may include shooting time parameters and shooting location parameters.
[0079] Determining the position parameters of the sun according to the shooting parameters may be determining the position parameters of the sun according to the shooting time parameters and the shooting location parameters of the remote sensing image; wherein the position parameters of the sun may include a sun altitude parameter and a sun azimuth parameter; as an example, the sun altitude parameter may be referred to as the sun altitude, which may be expressed as θ; the sun azimuth parameter may be referred to as the sun azimuth, which may be expressed as In practical applications, the solar altitude angle θ and solar azimuth angle at that time can be calculated or queried according to the time and place where the remote sensing image was taken.
[0080] Determining the sunlight incident angle image based on the normal parameters corresponding to the first object and the position parameters of the sun can be done by respectively determining a first direction vector and a second direction vector in a preset coordinate system based on the normal parameters corresponding to the first object and the position parameters of the sun; and determining the sunlight incident angle image based on the first direction vector and the second direction vector.
[0081] In an optional embodiment of the present invention, the position parameters of the sun include at least the altitude parameters of the sun and the azimuth parameters of the sun; determining the sunlight incident angle image based on the normal parameters corresponding to the first land object and the position parameters of the sun includes: determining a first direction vector of the normal parameters in a preset coordinate system according to the normal parameters corresponding to the first land object; determining a second direction vector of the incident light of the sun shining on the first land object in the preset coordinate system according to the position parameters of the sun; and determining the sunlight incident angle image based on the first direction vector and the second direction vector.
[0082] In this embodiment, the solar altitude angle parameter can be referred to as the solar altitude angle, which can be recorded as θ; the solar azimuth angle parameter can be referred to as the solar azimuth angle, which can be recorded as
[0083] Determine the first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first object; wherein the preset coordinate system can be determined according to actual conditions, which is not limited here. As an example, the preset coordinate system can be a ground object coordinate system, which can be referred to as the object coordinate system. Determine the first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first object can be determine the first direction vector of the normal parameter in the object coordinate system according to the normal parameter corresponding to the first object. For ease of understanding, an example is given here, in which the normal parameter can be a normal line, and the first direction vector of the normal parameter in the object coordinate system determined according to the normal parameter corresponding to the first object can be determine the direction vector of the normal line in the object coordinate system according to the normal line corresponding to the first object; the direction vector can be recorded as
[0084] Determining the second direction vector of the incident light from the sun on the first ground object in the preset coordinate system according to the altitude angle parameter of the sun and the azimuth angle parameter of the sun may be determining the second direction vector of the incident light from the sun on the first ground object in the object coordinate system according to the altitude angle parameter of the sun and the azimuth angle parameter of the sun. For ease of understanding, an example is given here, where the direction vector of the sun's rays in the object coordinate system is recorded as So:
[0085]
[0086]
[0087] z i = cosθ
[0088] Determining the sunlight incident angle image based on the first direction vector and the second direction vector can determine the cosine of the sunlight incident angle based on the normal direction vector and the sunlight direction vector, and use the cosine as the sunlight incident angle image. For ease of understanding, an example is given here, assuming that the sunlight incident angle is recorded as α, then the cosine calculation formula of the incident angle α is:
[0089]
[0090] In practical applications, the digital surface model can be used to calculate the normal direction of the ground object point corresponding to the pixel on the remote sensing image; the direction of the incident sunlight can be calculated based on the time and location of the image shooting; the solar incident angle of each pixel corresponding to the ground object point when the remote sensing image was shot can be calculated based on the normal and the direction of the incident sunlight, and the cosine of the angle can be used as the pixel value of the incident angle image to obtain an incident angle image that is spatially consistent with the remote sensing image. For ease of understanding, an example is given here. Figure 2 FIG. 1 is a flow chart of incident angle image construction in a remote sensing image classification method according to an embodiment of the present invention. Figure 2 shown.
[0091] The present invention mainly aims at the fact that the existing land object classification method based on remote sensing images does not take into account the incident direction of sunlight, resulting in insufficient ability of spectral features to identify land objects, which restricts the improvement of classification accuracy. To address this problem, the present invention designs a high-resolution remote sensing image classification method and system that takes into account the direction of incident sunlight in a three-dimensional scene, constructs an incident angle image as a feature map to participate in land object recognition, and improves the image classification accuracy.
[0092] In an optional embodiment of the present invention, the slicing processing of the remote sensing image, the sunlight incident angle image and the annotated image to obtain the slice image includes: obtaining a first slice image of a first preset size according to the remote sensing image, the sunlight incident angle image and the annotated image; cutting the first slice image into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
[0093] In this embodiment, the first slice image of the first preset size obtained according to the remote sensing image, the sunlight incident angle image and the annotated image may be a first slice image of the first preset size fixed to the remote sensing image, the sunlight incident angle image and the annotated image according to spatial consistency. The first preset size may be determined according to actual conditions and is not limited here. As an example, the first slice image may be a rectangular slice, and the first preset size may be 256×256. In practical applications, the remote sensing image, the incident angle image and the annotated image may be rectangular slices of a fixed size according to spatial consistency, and the slice image size is 256×256.
[0094] The first slice image is cut into a second preset size to obtain the slice image; wherein the second preset size can be determined according to actual conditions and is not limited here. As an example, the second preset size can be cut into 1 / 4 of the first slice image, that is, the slice image is cut into 64 pixels in both the horizontal and vertical directions.
[0095] The first slice image is cut into a second preset size to obtain the slice image mainly because in the convolutional neural network, zero padding of the edges during the convolution and pooling process will cause edge information distortion, so the classification accuracy of the semantic segmentation network for the image edge is often lower than that of the image interior. To address this problem, when slicing the image, it is cut at an interval of 64 pixels both horizontally and vertically, that is, 1 / 4 of the width of the slice image, to ensure that the image edge can appear inside another input image, thereby improving the semantic segmentation accuracy.
[0096] In an optional embodiment of the present invention, determining the remote sensing image classification model that takes sunlight into account based on the slice image and a preset convolutional neural network model includes: obtaining a training data set based on the slice image; and inputting the training data set into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account.
[0097] In this embodiment, obtaining the training data set according to the slice images may be randomly dividing the slice images into the training data set.
[0098] The training data set is input into the preset convolutional neural network model to determine that the remote sensing image classification model taking sunlight into account can be.
[0099] In an optional embodiment of the present invention, the training data set includes spectral images of a first preset threshold band, the sunlight incident angle images of a second preset threshold band and the annotated image in the remote sensing image, and the first preset threshold is greater than the second preset threshold; the step of inputting the training data set into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account comprises: inputting the spectral images of the first preset threshold band and the sunlight incident angle images of the second preset threshold band into the preset convolutional neural network model to obtain an output result; determining a loss parameter of the preset convolutional neural network model based on the output result and the annotated image; determining a correction value of a parameter of each layer in the preset convolutional neural network model based on the loss parameter; optimizing the preset convolutional neural network model according to the correction value until the loss parameter of the optimized preset convolutional neural network model is less than a third preset threshold, thereby obtaining the remote sensing image classification model that takes sunlight into account.
[0100] In this embodiment, the first preset threshold and the second preset threshold can be determined according to actual conditions and are not limited here. As an example, the first preset threshold can be 3, and the second preset threshold can be 1.
[0101] The spectral image of the first preset threshold bands and the sunlight incident angle image of the second preset threshold bands are input into the preset convolutional neural network model, and the output result can be obtained by inputting the spectral image of 3 bands and the incident angle image of 1 band into the neural network.
[0102] Determining the loss parameter of the preset convolutional neural network model according to the output result and the annotated image may be calculating the loss parameter of the preset convolutional neural network model according to the output result and the annotated image; in practical applications, the loss parameter may be referred to as loss for short.
[0103] Determining the correction value of each layer parameter in the preset convolutional neural network model based on the loss parameter may be calculating the correction value of each layer parameter in the preset convolutional neural network model through a back propagation algorithm based on the loss parameter.
[0104] Optimizing the preset convolutional neural network model according to the correction value until the loss parameter of the optimized preset convolutional neural network model is less than a third preset threshold value, obtaining the remote sensing image classification model taking sunlight into account can be optimizing the preset convolutional neural network model according to the correction value until the loss parameter of the optimized preset convolutional neural network model is less than a third preset threshold value, determining the target convolutional neural network model, and using the target convolutional neural network model as the remote sensing image classification model taking sunlight into account; wherein, the third preset threshold value can be determined according to actual conditions and is not limited here. In practical applications, the third preset threshold value is generally a given threshold value.
[0105] In practical applications, the training data set may also be referred to as a training sample. In this embodiment, a preset convolutional neural network is trained using the training data set to obtain a remote sensing image classification model that takes into account the direction of incident sunlight. Specifically, the training data set includes: inputting spectral images of three bands and incident angle images of one band in the training sample into the neural network to obtain output results; calculating the loss of the model based on the output results and the labeled images in the sample; calculating the correction values of the parameters of each layer of the network through a back propagation algorithm based on the model loss; and continuously repeating the above steps until the model loss is less than a given threshold, thereby obtaining a remote sensing image classification model that takes into account the direction of incident sunlight.
[0106] In an optional embodiment of the present invention, the method further includes: obtaining a verification data set based on the slice image; and performing accuracy evaluation on the remote sensing image classification model that takes sunlight into account based on the verification data set.
[0107] In this embodiment, obtaining the verification data set according to the slice images may be randomly dividing the slice images into the verification data set.
[0108] The accuracy of the remote sensing image classification model taking sunlight into consideration may be evaluated based on the validation data set.
[0109] In an optional embodiment of the present invention, the accuracy assessment of the remote sensing image classification model taking sunlight into account based on the verification data set includes: inputting the verification data set into the remote sensing image classification model taking sunlight into account to obtain a classification result; determining a target classification result corresponding to a pixel with the highest overlap in the classification result; and performing an accuracy assessment on the remote sensing image classification model taking sunlight into account based on the target classification result.
[0110] In this embodiment, the verification data set is input into the remote sensing image classification model taking sunlight into account, and the classification result can be obtained by inputting the spectral image of the fourth preset threshold band and the sunlight incident angle image of the fifth preset threshold band in the verification data set into the remote sensing image classification model taking sunlight into account to obtain the classification result; wherein, the fourth preset threshold and the fifth preset threshold can be determined according to actual conditions, and are not limited here. As an example, the fourth preset threshold can be 3, and the fifth preset threshold can be 1. In actual applications, the spectral images of 3 bands and the incident angle image of 1 band in the verification data can be input into a trained neural network to obtain the classification result.
[0111] The target classification result corresponding to the pixel with the highest degree of overlap in the classification result is determined mainly because the first slice image is cut according to 1 / 4 of the slice image width when slicing, so there is overlap between adjacent slices. In practical applications, a pixel may appear in multiple slice images and obtain multiple classification results. If the classification results of a pixel in multiple images are not completely consistent, the category with the most classification results is selected as the final category of the pixel, that is, the target classification result.
[0112] According to the target classification result, the accuracy assessment of the remote sensing image classification model taking sunlight into account can be performed by comparing the target classification result with the labeled images in the verification data set, calculating the overall accuracy of the remote sensing image classification model taking sunlight into account, and performing the accuracy assessment based on the overall accuracy. As an example, the overall accuracy of the remote sensing image classification model taking sunlight into account can be calculated by dividing the number of correctly classified pixels by the number of all pixels. If the overall accuracy reaches a set threshold, it is considered that the classification model meets the accuracy standard and can be used for remote sensing image classification.
[0113] In this embodiment, considering that the traditional network for semantic segmentation does not take into account the incident angle information of the ground object, to address this problem, the present invention designs a convolutional neural network that can take into account the incident angle information of light, which is used to extract the composite features of the incident angle of sunlight and spectral information of the ground object, thereby improving the accuracy of image classification.
[0114] For ease of understanding, the present invention illustrates a remote sensing image classification method as a high-resolution remote sensing image classification method that takes into account the direction of incident sunlight in a three-dimensional scene, and the steps are as follows:
[0115] Step 1: Generation of incident angle image.
[0116] The incident angle of sunlight relative to a certain point is the angle between the incident light direction and the normal direction of the point, and the cosine value of the angle is used as the pixel value of the incident angle image. Figure 2 As shown, firstly, the normal direction is calculated based on the digital surface model, which can be obtained by dense matching of LIDAR point cloud or multi-view images. According to the time and place of remote sensing image shooting, the local solar altitude angle θ and solar azimuth angle can be calculated or queried. Let the direction vector of the sun's rays in the object coordinate system be So:
[0117]
[0118]
[0119] z i = cosθ
[0120] Let the direction vector of the normal line in the object coordinate system be Then the formula for calculating the cosine of the incident angle α is:
[0121]
[0122] Step 2: Dataset construction.
[0123] Different types of ground objects in some high-resolution remote sensing images are manually annotated, and annotated images with consistent spatial relationships with the original image pixels are generated. The pixel values on the annotated images represent the types of ground objects.
[0124] The remote sensing image, incident angle image and annotated image are sliced into fixed-size rectangles according to spatial consistency, and the slice image size is 256×256. Since zero-filling the edges during the convolution and pooling process will cause edge information distortion, the classification accuracy of the semantic segmentation network for the image edge is often lower than that for the image interior. To address this problem, when slicing the image, it is cut at an interval of 64 pixels horizontally and vertically, that is, 1 / 4 of the width of the slice image, to ensure that the image edge can appear inside another input image, thereby improving the semantic segmentation accuracy. Finally, the slice image is randomly divided into a training data set and a validation data set.
[0125] Step 3: Build a convolutional neural network.
[0126] The relationship between the solar incident angle, spectral value and ground object category is very complex, and it is difficult to obtain a complete physical model. Therefore, the present invention uses a deep learning method to mine the mathematical relationship between them from a large amount of sample data, and models the relationship using a convolutional neural network. The implementation method based on deep learning can automatically extract features for identifying ground objects from optical image data and incident angle images, providing greater flexibility and a wider range of application scenarios for the present invention.
[0127] The present invention uses the DeepLabv3+ network as the basic network framework for remote sensing image semantic segmentation. The difference is that the DeepLabv3+ network inputs 3-band visible light images and does not consider the incident angle image. To address this problem, the present invention modifies the DeepLabv3+ network input layer to a 4-band input, where 3 bands are the 3 spectral bands of the remote sensing image and 1 band is the incident angle image.
[0128] Step 4: Network training.
[0129] The convolutional neural network constructed in the previous step is trained using the training data set to obtain a remote sensing image classification model that takes into account the direction of solar incident light. Specifically, the following steps are used: the spectral images of three bands and the incident angle images of one band in the training samples are input into the neural network to obtain the output results; the loss of the model is calculated based on the output results and the labeled images in the samples; the correction values of the parameters of each layer of the network are calculated through the back propagation algorithm based on the model loss; the above steps are repeated until the model loss is less than the given threshold, and the remote sensing image classification model that takes into account the direction of solar incident light can be obtained.
[0130] Step 5: Model validation.
[0131] The validation data set was used to test the classification model and evaluate its accuracy, including: inputting the spectral images of three bands and the incident angle image of one band in the test data set into the trained neural network and obtaining the classification results; since in the second step of data set construction, the image was sliced according to 1 / 4 of the slice image width, there was overlap between adjacent slices, that is, a pixel may appear in multiple slice images, and multiple classification results were obtained. If the classification results of a pixel in multiple images were not completely consistent, the category with the most classification results was selected as the final category of the pixel; the final classification result was compared with the real annotated image of the validation set, and the overall accuracy was calculated by dividing the number of correctly classified pixels by the number of all pixels. If the overall accuracy reached the set threshold, it was considered that the classification model had met the accuracy standards and could be used for remote sensing image classification.
[0132] For ease of understanding, Figure 3 This is another implementation flow diagram of a remote sensing image classification method according to an embodiment of the present invention. Considering that the existing land object classification method based on remote sensing images does not take into account the incident direction of sunlight, the spectral characteristics are insufficient for land object recognition, which restricts the improvement of classification accuracy. In response to this problem, the present invention designs a high-resolution remote sensing image classification method and system that takes into account the direction of the sun's incident light in a three-dimensional scene, constructs an incident angle image as a feature map to participate in land object recognition, and improves image classification accuracy. And the traditional semantic segmentation network does not take into account the incident angle information of the land object. In response to this problem, the present invention designs a convolutional neural network that can take into account the incident angle information of light, which is used to extract the composite features of the incident angle of the sun's light and the spectral information of the land object, and improves the image classification accuracy.
[0133] The three-dimensional surface model of the earth's surface can be used to calculate the normal direction of each ground point, and then the direction of the incident light can be calculated. Then, the deep neural network can be used to mine the statistical relationship between the spectral value and the direction of the incident light, and extract features with higher ability to identify ground objects.
[0134] Compared with the existing technology at this stage, the present invention has the following advantages and effects:
[0135] First, the present invention constructs incident angle images to participate in remote sensing image classification, thereby solving the problem of differences in spectral characteristics caused by different direct solar energy received at different positions on the surface of three-dimensional objects. The designed remote sensing image classification method and system can effectively improve the accuracy of object recognition.
[0136] Second, traditional convolutional neural networks cannot take into account incident angle information. The present invention improves the DeepLabv3+ deep neural network for remote sensing image semantic segmentation to extract composite high-level features of remote sensing images and incident angle images in the training data set, thereby improving the accuracy of ground object recognition.
[0137] This embodiment provides a remote sensing image classification device. Figure 4 FIG. 1 is a schematic diagram of the structure of a remote sensing image classification device according to an embodiment of the present invention. Figure 4 As shown, the device 200 includes: an obtaining unit 201, a processing unit 202, a determining unit 203 and a classification unit 204, wherein:
[0138] The obtaining unit 201 is used to obtain a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image;
[0139] The processing unit 202 is used to perform slice processing on the remote sensing image, the sunlight incident angle image, and the annotated image to obtain a slice image;
[0140] The determination unit 203 is used to determine a remote sensing image classification model taking sunlight into consideration according to the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image;
[0141] The classification unit 204 is used to classify the ground objects in the remote sensing image based on the remote sensing image classification model taking sunlight into consideration.
[0142] In other embodiments, the acquisition unit 201 is also used to determine the normal parameters corresponding to the first ground object in the remote sensing image based on the three-dimensional surface model of the earth; obtain the shooting parameters of the remote sensing image, and determine the position parameters of the sun based on the shooting parameters; and determine the sunlight incident angle image based on the normal parameters corresponding to the first ground object and the position parameters of the sun.
[0143] In other embodiments, the position parameters of the sun include at least an altitude parameter of the sun and an azimuth parameter of the sun; the acquisition unit 201 is further used to determine a first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first land object; determine a second direction vector of the incident light of the sun shining on the first land object in the preset coordinate system according to the position parameters of the sun; and determine the sunlight incident angle image based on the first direction vector and the second direction vector.
[0144] In other embodiments, the processing unit 202 is further used to obtain a first slice image of a first preset size based on the remote sensing image, the sunlight incident angle image and the annotated image; cut the first slice image into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
[0145] In other embodiments, the determination unit 203 is further used to obtain a training data set based on the slice image; and input the training data set into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account.
[0146] In other embodiments, the training data set includes spectral images of a first preset threshold band, the sunlight incident angle images of a second preset threshold band, and the annotated images in the remote sensing image, and the first preset threshold is greater than the second preset threshold; the determination unit 203 is also used to input the spectral images of the first preset threshold band and the sunlight incident angle images of the second preset threshold band into the preset convolutional neural network model to obtain an output result; determine the loss parameters of the preset convolutional neural network model according to the output result and the annotated image; determine the correction values of the parameters of each layer in the preset convolutional neural network model based on the loss parameters; optimize the preset convolutional neural network model according to the correction values until the loss parameters of the optimized preset convolutional neural network model are less than the third preset threshold, so as to obtain the remote sensing image classification model that takes sunlight into account.
[0147] In other embodiments, the device 200 further includes an evaluation unit.
[0148] The obtaining unit 201 is further used to obtain a verification data set according to the slice image;
[0149] The evaluation unit is used to evaluate the accuracy of the remote sensing image classification model taking sunlight into account based on the verification data set.
[0150] In other embodiments, the evaluation unit is further used to input the verification data set into the remote sensing image classification model that takes sunlight into account to obtain a classification result; determine a target classification result corresponding to a pixel with the highest overlap in the classification result; and perform an accuracy evaluation on the remote sensing image classification model that takes sunlight into account based on the target classification result.
[0151] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present invention, please refer to the description of the method embodiment of the present invention for understanding.
[0152] It should be noted that, in the embodiment of the present invention, if the above-mentioned remote sensing image classification method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical embodiment of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a remote sensing image classification device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.
[0153] Correspondingly, an embodiment of the present invention provides a remote sensing image classification device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, it implements the steps in a remote sensing image classification method provided in the above embodiment.
[0154] Correspondingly, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the remote sensing image classification method provided in the above embodiment are implemented.
[0155] It should be noted that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of the present invention, please refer to the description of the method embodiments of the present invention for understanding.
[0156] It should be noted that Figure 5 FIG. 1 is a schematic diagram of a hardware entity structure of a remote sensing image classification device according to an embodiment of the present invention. Figure 5 As shown, the hardware entity of the remote sensing image classification device 300 includes: a processor 301 and a memory 303 . Optionally, the remote sensing image classification device 300 may also include a communication interface 302 .
[0157] It can be understood that the memory 303 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 303 described in the embodiment of the present invention is intended to include but is not limited to these and any other suitable types of memories.
[0158] The method disclosed in the above embodiment of the present invention can be applied to the processor 301, or implemented by the processor 301. The processor 301 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 301 or the instruction in the form of software. The above processor 301 can be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 301 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined to execute. The software module can be located in a storage medium, which is located in the memory 303. The processor 301 reads the information in the memory 303 and completes the steps of the above method in combination with its hardware.
[0159] In an exemplary embodiment, the classification device of the remote sensing image can be implemented by one or more application-specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general-purpose processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.
[0160] In the several embodiments provided by the present invention, it should be understood that the disclosed methods and devices can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another observation, or some features can be ignored or not executed. In addition, the communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0161] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.
[0162] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), disks or optical disks, etc. Various media that can store program codes.
[0163] Alternatively, if the above-mentioned integrated unit of the embodiment of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical embodiment of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a remote sensing image classification device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.
[0164] The present invention is based on the method, device and computer storage medium for determining the quality of satellite observation quantities recorded in the examples. The embodiments of the present invention are taken as examples only, but are not limited to this. As long as it involves the method, device and computer storage medium for determining the quality of satellite observation quantities, they are within the protection scope of the present invention.
[0165] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.
[0166] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0167] The above is only an embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A classification method for remote sensing images, It is characterized in that The method comprises: Obtaining a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image; Slicing the remote sensing image, the sunlight incident angle image, and the annotated image to obtain a slice image; Determining a remote sensing image classification model that takes sunlight into account according to the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image; Classifying the ground objects in the remote sensing image based on the remote sensing image classification model taking sunlight into account; Wherein, obtaining the sunlight incident angle image of the first object in the remote sensing image includes: Determining normal parameters corresponding to the first ground object in the remote sensing image according to the three-dimensional surface model of the ground; Obtaining shooting parameters of the remote sensing image, and determining position parameters of the sun according to the shooting parameters; Determine the sunlight incident angle image based on the normal parameter corresponding to the first ground object and the position parameter of the sun; The sun's position parameters at least include a sun's altitude parameter and a sun's azimuth parameter; and determining the sunlight incident angle image based on the normal parameter corresponding to the first ground object and the sun's position parameters includes: Determine a first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first ground object; Determine a second direction vector of the incident light from the sun on the first ground object in the preset coordinate system according to the altitude angle parameter of the sun and the azimuth angle parameter of the sun; The sunlight incident angle image is determined based on the first direction vector and the second direction vector.
2. The method according to claim 1, It is characterized in that The step of performing slicing processing on the remote sensing image, the sunlight incident angle image and the annotated image to obtain a slice image includes: Obtaining a first slice image of a first preset size according to the remote sensing image, the sunlight incident angle image and the annotated image; The first slice image is cut into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
3. The method according to claim 2, It is characterized in that The step of determining a remote sensing image classification model taking sunlight into consideration based on the slice image and a preset convolutional neural network model comprises: Obtaining a training data set according to the slice image; The training data set is input into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account.
4. The method according to claim 3, It is characterized in that The training data set includes spectral images of a first preset threshold number of bands in the remote sensing image, the sunlight incident angle images of a second preset threshold number of bands, and the labeled images, wherein the first preset threshold is greater than the second preset threshold; and inputting the training data set into the preset convolutional neural network model to determine the remote sensing image classification model taking sunlight into account includes: Inputting the spectral image of the first preset threshold wavebands and the sunlight incident angle image of the second preset threshold wavebands into the preset convolutional neural network model to obtain an output result; Determining a loss parameter of the preset convolutional neural network model according to the output result and the annotated image; Determining a correction value of each layer parameter in the preset convolutional neural network model based on the loss parameter; The preset convolutional neural network model is optimized according to the correction value until the loss parameter of the optimized preset convolutional neural network model is less than a third preset threshold, thereby obtaining the remote sensing image classification model that takes sunlight into account.
5. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: Obtaining a verification data set according to the slice images; The accuracy of the remote sensing image classification model taking sunlight into account is evaluated based on the verification data set.
6. The method according to claim 5, It is characterized in that The step of evaluating the accuracy of the remote sensing image classification model taking sunlight into account based on the verification data set includes: Inputting the verification data set into the remote sensing image classification model taking sunlight into account to obtain a classification result; Determine the target classification result corresponding to the pixel with the highest overlap in the classification result; The accuracy of the remote sensing image classification model taking sunlight into account is evaluated according to the target classification result.
7. A remote sensing image classification device, It is characterized in that The device comprises: an obtaining unit, a processing unit, a determining unit and a classification unit, wherein: The obtaining unit is used to obtain a labeled image and a sunlight incident angle image of a first object in the remote sensing image; the first object is any object in the remote sensing image; The processing unit is used to perform slice processing on the remote sensing image, the sunlight incident angle image, and the annotated image to obtain a slice image; The determination unit is used to determine a remote sensing image classification model that takes sunlight into consideration according to the slice image and a preset convolutional neural network model; the preset convolutional neural network model is determined by a preset remote sensing image and a preset sunlight incident angle image; The classification unit is used to classify the ground objects in the remote sensing image based on the remote sensing image classification model taking sunlight into account; The obtaining unit is further used to determine the normal parameters corresponding to the first ground object in the remote sensing image according to the three-dimensional surface model of the ground; obtain the shooting parameters of the remote sensing image, and determine the position parameters of the sun according to the shooting parameters; determine the sunlight incident angle image based on the normal parameters corresponding to the first ground object and the position parameters of the sun; The position parameters of the sun include at least the altitude parameters and the azimuth parameters of the sun; the acquisition unit is further used to determine the first direction vector of the normal parameter in a preset coordinate system according to the normal parameter corresponding to the first object; determine the second direction vector of the incident light of the sun shining on the first object in the preset coordinate system according to the position parameters of the sun; and determine the sunlight incident angle image based on the first direction vector and the second direction vector.
8. The device according to claim 7, It is characterized in that The processing unit is further used to obtain a first slice image of a first preset size based on the remote sensing image, the sunlight incident angle image and the annotated image; cut the first slice image into a second preset size to obtain the slice image; the second preset size is smaller than the first preset size.
9. The device according to claim 8, It is characterized in that The determination unit is further used to obtain a training data set based on the slice image; and input the training data set into the preset convolutional neural network model to determine the remote sensing image classification model that takes sunlight into account.
10. The device according to claim 9, It is characterized in that The training data set includes spectral images of a first preset threshold band, the sunlight incident angle images of a second preset threshold band and the annotated images in the remote sensing image, and the first preset threshold is greater than the second preset threshold; the determination unit is further used to input the spectral images of the first preset threshold band and the sunlight incident angle images of the second preset threshold band into the preset convolutional neural network model to obtain an output result; determine the loss parameters of the preset convolutional neural network model according to the output result and the annotated image; and determine the correction values of the parameters of each layer in the preset convolutional neural network model based on the loss parameters; The preset convolutional neural network model is optimized according to the correction value until the loss parameter of the optimized preset convolutional neural network model is less than a third preset threshold, thereby obtaining the remote sensing image classification model that takes sunlight into account.
11. The device according to any one of claims 7 to 10, It is characterized in that The device further comprises an evaluation unit, The obtaining unit is further used to obtain a verification data set according to the slice image; The evaluation unit is used to evaluate the accuracy of the remote sensing image classification model taking sunlight into account based on the verification data set.
12. The device according to claim 11, It is characterized in that The evaluation unit is further used to input the verification data set into the remote sensing image classification model taking sunlight into account to obtain a classification result; and determine a target classification result corresponding to a pixel with the highest overlap in the classification result; The accuracy of the remote sensing image classification model taking sunlight into account is evaluated according to the target classification result.
13. A remote sensing image classification device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, It is characterized in that When the processor executes the program, the steps in the method according to any one of claims 1 to 6 are implemented.
14. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 6 are implemented.
