A sample data migration enhancement method and system based on remote sensing image spatiotemporal feature correlation fusion

Abstract

The present invention proposes a sample data migration and enhancement method and system based on the correlation and fusion of remote sensing image spatiotemporal spectral features. The method is: obtaining multispectral remote sensing images and digital elevation model data of the target area; calculating the normalized vegetation index of the target area based on the multispectral remote sensing images, and calculating the slope data of the target area based on the digital elevation model data; vertically correlating and combining the multispectral remote sensing images, digital elevation model data, normalized vegetation index and slope data of the target area according to geographic coordinates to form a data cube; marking the basic sample data of typical landforms in the data cube; combining the regional growing algorithm and the Mahalanobis distance algorithm to migrate and enhance the marked basic sample data of typical landforms. The method can provide new technologies and methods for the production, enhancement and automatic generation of large-scale sample data sets.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing monitoring and identification of geological disasters, and in particular to a sample data migration enhancement method and system based on remote sensing image spatiotemporal feature correlation fusion. Background Art

[0002] The main geological disasters in the world include collapse, debris flow, landslide, ground subsidence, ground fissures, etc. Among them, landslide is one of the most harmful and influential types of geological disasters.

[0003] Mountainous areas, with high mountains, deep valleys and complex geology, are home to a large number of high-risk landslides. Under the influence of inducing factors such as earthquakes or heavy rains, landslide disasters will occur frequently on a large scale. Therefore, the rapid and accurate extraction of important disaster information such as the location, number, scope and distribution of landslides has important theoretical significance and practical value in emergency rescue, disaster assessment and disaster prevention and mitigation.

[0004] The development of optical remote sensing imaging technology and image processing technology based on satellite or aerial platforms has brought new technical means for large-scale monitoring and accurate extraction of landslide disaster information. Many experts and scholars at home and abroad have carried out a lot of landslide feature interpretation and remote sensing identification based on optical remote sensing images and machine learning methods. However, whether it is traditional statistical analysis methods, classical machine learning methods, or the current mainstream deep learning methods, a large amount of sample data is required. The production of high-quality sample data sets often consumes a lot of manpower, material resources and financial resources. In addition, the lack of training sample data often causes underfitting problems in machine learning models during training, which in turn affects the target recognition accuracy of machine learning models. Therefore, how to produce large-scale sample data sets at low cost and high efficiency has become a problem that needs to be solved urgently. Summary of the invention

[0005] In order to overcome the defects existing in the above-mentioned prior art, the purpose of the present invention is to provide a sample data migration enhancement method and system based on the correlation fusion of spatiotemporal and spectral features of remote sensing images.

[0006] In order to achieve the above-mentioned object of the present invention, the present invention provides a sample data migration enhancement method based on remote sensing image spatiotemporal feature correlation fusion, comprising the following steps:

[0007] Obtain multispectral remote sensing images and digital elevation model data of the target area before and after the disaster, preprocess them, and convert the multispectral remote sensing images into multispectral reflectance images;

[0008] Calculating a surface normalized vegetation index of a target area based on the multispectral reflectance image, and calculating slope data of the target area based on the digital elevation model data;

[0009] The multispectral reflectance images, digital elevation model data, surface normalized vegetation index and slope data of the target area before and after the disaster are vertically correlated and combined according to geographic coordinates to form a data cube that integrates the spatiotemporal spectral characteristic information of landslides and typical background objects;

[0010] Marking basic sample data of typical land features in the data cube;

[0011] The region growing algorithm and Mahalanobis distance algorithm are combined to migrate and enhance the labeled basic sample data of typical land features.

[0012] This method can perform data migration enhancement on the basic sample data set manually labeled, thereby alleviating the underfitting problem encountered in the training process of the machine learning model. The new sample data set obtained after data enhancement by the present invention can be used to train the classic support vector machine model and the current popular convolutional neural network model, which can effectively improve the recognition accuracy of various machine learning models for typical ground objects; it can provide new technologies and methods for the production, enhancement and automatic generation of large-scale sample data sets.

[0013] In an optional solution, the ratio of the sum of the reflectance image of the near infrared band and the reflectance image of the red band in the multispectral reflectance image of the target area and the difference between the two is used as the surface normalized vegetation index of the target area.

[0014] In an optional solution, the steps for calculating the slope data are:

[0015] Given a pixel (i, j) on the digital elevation model data, i and j are the row and column numbers of the pixel respectively;

[0016] Select n×n neighboring pixel areas around pixel (i, j) and calculate the average height change rate of pixel (i, j) in the x direction, where n is a positive integer:

[0017]

[0018] Among them, cellsize x is the spatial width of a single pixel in the x direction, and h(i,j) is the altitude of the pixel (i,j);

[0019] Calculate the average height change rate of pixel (i, j) in the y direction:

[0020]

[0021] Among them, cellsize y is the spatial width of a single pixel in the y direction;

[0022] Calculate the slope of pixel (i,j):

[0023]

[0024] The n×n pixel window area is moved to traverse each pixel, and the slope value of each pixel in the entire target area is calculated to obtain the slope data of the target area.

[0025] Slope is the steepness of a certain pixel point on the surface. It is an important quantitative indicator of the spatial characteristics of the terrain in the landslide occurrence area. It is usually expressed in angle or percentage. The occurrence of landslides is closely related to the slope of the terrain. Based on this optional solution, the slope data of the target area can be obtained quickly and accurately.

[0026] In an optional solution, the steps of migrating and enhancing the labeled basic sample data of typical objects are as follows:

[0027] The unlabeled sample points are searched around the labeled sample points by using the region growing algorithm. When the Mahalanobis distance between the labeled sample points and the unlabeled sample points is less than the growth threshold T M When the unlabeled sample points are used as sample points of the same type as the labeled sample points, typical sample data migration enhancement is achieved.

[0028] This optional solution uses a region growing algorithm to expand the search for new similar sample points, and introduces the Mahalanobis distance to measure the similarity of each sample point in a high-dimensional space, thereby achieving accurate search and migration enhancement of similar sample points.

[0029] In an optional solution, the Mahalanobis distance calculation formula is:

[0030]

[0031] Where Y = [y 1 ,y 2 ,",y i ,...,y p ] T , is a set of unlabeled sample points, y i represents the multispectral reflectance, digital elevation model data, surface normalized vegetation index and slope data of the i-th sample point in Y, is a set of labeled sample points, x i represents the multispectral reflectance, digital elevation model data, surface normalized vegetation index and slope data of the i-th sample point in X, c represents the labeled sample category, Σ c is the covariance matrix of the multidimensional features of the c-th class of labeled sample data sets, D M (Y,X c ) is used to characterize the similarity between labeled sample points and unlabeled sample points.

[0032] In an optional solution, the steps of typical sample data migration enhancement are:

[0033] Step 1: Scan the labeled samples in sequence and take the first labeled basic sample as the central pixel (i 0 ,j 0 );

[0034] Step 2: Take the center pixel (i 0 ,j 0 ) as the center, considering the central pixel (i 0 ,j 0 ) in the surrounding 8-connected neighborhoods. If there is no pixel (i, j) marked with the category it belongs to, then if the pixel (i, j) is 0 ,j 0 ) is less than the growth threshold T M When , the growth criterion is met, and the pixel (i, j) and the center pixel (i 0 ,j 0 ) are merged as the same type of features in the same area, and the pixel (i, j) is pushed into the stack;

[0035] Step 3: Take a pixel from the stack and use it as the center pixel (i 0 ,j 0 ) Return to step 2;

[0036] Step 4: When the stack is empty or the total number of samples in the stack reaches the upper limit T of a single sample growth num_max When , return to step 1 and take one of the remaining labeled basic samples as the central pixel (i 0 ,j 0 );

[0037] Step 5: Repeat steps 1-4 until all marked basic sample points are traversed and the growth is completed.

[0038] This optional solution can traverse all marked sample points, and can perform accurate search and migration enhancement on all marked sample points of the same type.

[0039] The present invention also proposes a sample data migration enhancement system, which includes a processing module, a storage module, and a data acquisition module for acquiring multispectral remote sensing images and digital elevation model data of a target area;

[0040] The data acquisition module is connected to the processing module and sends the multispectral remote sensing image and digital elevation model data to the processing module; the processing module is communicatively connected to the storage module, and the storage module is used to store at least one executable instruction, and the executable instruction enables the processing module to perform operations corresponding to the sample data migration enhancement method based on the correlation and fusion of spatiotemporal spectral features of remote sensing images based on the multispectral remote sensing image and digital elevation model data, thereby performing migration enhancement on the sample data.

[0041] The sample data migration enhancement system has all the advantages of the above-mentioned sample data migration enhancement method based on the correlation fusion of spatiotemporal and spectral features of remote sensing images.

[0042] The present invention also proposes a target recognition method, which obtains samples based on the sample data migration enhancement method based on the remote sensing image spatiotemporal spectral feature correlation fusion mentioned above;

[0043] Construct a machine learning model, use the samples to train the machine learning model, and use the trained machine learning model to perform target recognition on the samples to be recognized.

[0044] The beneficial effects of the present invention are:

[0045] The present invention integrates multi-dimensional information such as spectral characteristics, temporal variation characteristics, and terrain spatial characteristics involved in landslide remote sensing monitoring and identification to construct a data cube of typical landforms, providing basic supporting data for the identification of landslides and typical background landforms.

[0046] The present invention adopts a region growing algorithm to expand the search for new similar sample points, and introduces the Mahalanobis distance to measure the similarity of each sample point in a high-dimensional space, thereby achieving accurate search and migration enhancement of similar sample points.

[0047] The present invention calculates the Mahalanobis distance between the marked sample point and each sample point in the surrounding 8-connected neighborhood in high-dimensional space, and uses it as the growth criterion of the region growing algorithm to achieve accurate search and migration amplification of similar sample points. The Mahalanobis distance can eliminate the influence of the dimension on the feature, has translation invariance in the high-dimensional feature space, and takes into account the correlation of its own features. Therefore, compared with the traditional Euclidean distance or spectral angle, it can more accurately characterize the similarity between two sample points.

[0048] The present invention can perform data enhancement on a small number of high-quality labeled sample points, effectively expand the basic sample data set, and thus alleviate the underfitting problem encountered in the training process of the machine learning model. The new sample data set obtained after data enhancement using the patented method can be used to train the classic support vector machine (SVM) model and the currently popular convolutional neural network (CNN) model, which can effectively improve the recognition accuracy of various machine learning models for typical ground objects.

[0049] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 It is a schematic diagram of the process of the present invention;

[0052] Figure 2 It is a schematic diagram of the 8-connected neighborhood around the central pixel, where the five-pointed star is the central pixel;

[0053] Figure 3a It is the surface reflectivity image of the target area after the earthquake landslide;

[0054] Figure 3b It is the surface reflectivity image of the target area before the earthquake landslide;

[0055] Figure 4 It is the digital elevation model data of the target area;

[0056] Figure 5a It is a schematic diagram of the distribution of the surface normalized difference vegetation index in the target area after the disaster;

[0057] Figure 5b It is a schematic diagram of the distribution of the surface normalized difference vegetation index in the target area before the disaster;

[0058] Figure 6 is the slope data map of the target area;

[0059] Figure 7 It is a data cube formed by combining multi-dimensional feature information of the target area;

[0060] Figure 8 This is the manually marked landslide and typical background feature sample data, where red is the landslide sample; blue is the vegetation sample; yellow is the water sample; green is the urban construction area and cloud sample;

[0061] Figure 9a is the distribution of landslide and typical background features samples before data migration and enhancement;

[0062] Figure 9b The distribution of landslide and typical background features samples after data migration enhancement based on region growing algorithm and Euclidean distance;

[0063] Fig.9c The distribution of landslide and typical background features samples after data migration enhancement based on region growing algorithm and Mahalanobis distance;

[0064] Among them, red is landslide samples; blue is vegetation samples; yellow is water samples; green is urban construction area and cloud samples;

[0065] Fig.10 The basic structure of the CNN model used in the experiment in Example 1;

[0066] Fig.11a This is a schematic diagram of the classification results after training the SVM model based on the manually annotated basic sample data set;

[0067] Fig.11b This is a schematic diagram of the classification results after the SVM model is trained using the region growing algorithm and Mahalanobis distance to perform data migration enhancement on the sample data set;

[0068] Fig.11c This is a schematic diagram of the classification results after training the CNN model based on the manually annotated basic sample data set;

[0069] Fig.11d Schematic diagram of the classification results after training the CNN model using the sample data set with region growing algorithm and Mahalanobis distance for data migration enhancement. DETAILED DESCRIPTION

[0070] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0071] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0072] Embodiment 1

[0073] This embodiment uses the multispectral images of the Luding 6.8-magnitude earthquake acquired by the Sentinel-2A / B satellite and the digital elevation model (DEM) data acquired by the ALOS (Advanced Land Observing Satellite) satellite phased array L-band synthetic aperture radar (PALSAR) to carry out research on the sample data migration enhancement method based on the spatial-temporal spectral feature association fusion of landslide remote sensing images. Specifically, it refers to the automatic generation and migration enhancement of typical target sample data such as landslides, vegetation, water bodies and man-made objects in multispectral remote sensing images, and conducts application experiments on the enhanced sample data based on the SVM model and the CNN model. The relevant experimental results show that the training of SVM and CNN models using sample data processed by spatial-temporal spectral feature association fusion and migration enhancement can effectively alleviate the underfitting problem encountered in the model training process and improve its recognition accuracy for typical objects.

[0074] like Figure 1 As shown, the specific steps include:

[0075] The first step: data selection and acquisition, that is, obtaining multispectral remote sensing images and digital elevation model data of the target area.

[0076] In order to improve the accuracy of remote sensing monitoring and identification of landslides, basic training sample data are established based on multi-temporal, multi-spectral remote sensing images and digital elevation model data before and after the landslide occurs. Among them, multi-spectral remote sensing images after the landslide occurs (after the disaster) should at least contain visible light band images within the three bands of blue light (450-495nm), green light (495-570nm), and red light (620-780nm), as well as images within one or more near-infrared bands (780-1200nm); multi-spectral remote sensing images before the landslide occurs (before the disaster) are generally historical images of the corresponding area, and their spectral band settings should be the same or similar to the post-disaster data. Digital elevation model data can be formed based on the previous on-site measurement data of the target area, lidar remote sensing data, photogrammetry solution data, or synthetic aperture radar (SAR) interferometric measurement data.

[0077] This embodiment selects the river valley of Moshi Town, which is about 10 km away from the epicenter of the 6.8-magnitude Luding earthquake, as the experimental area. Its spatial range is roughly a rectangular area with the upper left corner: E102.08°, N29.68°; the lower right corner: E102.18°, N29.59°. The post-disaster remote sensing image of the experimental area is a multispectral image of the area acquired by the Sentinel-2A satellite on October 1, and the pre-disaster remote sensing image is a multispectral image of the area acquired by the Sentinel-2B satellite on August 7. The corresponding bands are shown in Table 1. The spatial resolution of the digital elevation model data acquired by the ALOS satellite PALSAR is 12.5m.

[0078] Table 1 Central wavelength and spatial resolution of multispectral remote sensing images from Sentinel-2 satellite

[0079]

[0080]

[0081] Step 2: Data Preprocessing

[0082] The multispectral remote sensing images are preprocessed by radiation correction, atmospheric correction and orthorectification, and converted into surface reflectance images based on geographic coordinate system. The digital elevation model data are resampled to the same spatial resolution as the optical remote sensing images by bilinear interpolation method, and converted into the same geographic coordinate system as the optical remote sensing images.

[0083] In this embodiment, the L1C-level multispectral images acquired by the Sentinel-2A / B satellite are preprocessed by radiation correction, atmospheric correction, and orthorectification to form a surface reflectance image with a spatial resolution of 10m based on the WGS-84 geographic coordinate system, such as Figure 3a and Figure 3b The 12.5m digital elevation model data is resampled to 10m spatial resolution using bilinear interpolation and converted to the WGS-84 geographic coordinate system, as shown in Figure 4 shown.

[0084] Step 3: Auxiliary data layer generation

[0085] The auxiliary data layers here include the surface Normalized Difference Vegetation Index (NDVI) of the target area calculated based on the multispectral reflectance image and the slope data of the target area calculated based on the digital elevation model data.

[0086] (1) Generation of auxiliary data layers for surface normalized difference vegetation index

[0087] The surface normalized vegetation index is the ratio of the sum of the reflectance image of the near-infrared band and the reflectance image of the red band in the multispectral reflectance image of the target area to the difference between the two. From this, the surface normalized vegetation index auxiliary data layer can be generated.

[0088] In this embodiment, the surface normalized vegetation index B of the entire target area is calculated using the multispectral images before and after the disaster according to the following formula: NDVI :

[0089]

[0090] Among them, B NIR is the reflectance image in the near-infrared band; B R is the reflection image of the red light band. The surface normalized vegetation index of the target area before and after the disaster is calculated, such as Figure 5a and Figure 5b As shown in Figure 1, the NDVI usually ranges from -1 to 1. A positive NDVI value (close to 1) indicates strong vegetation cover, while a value less than zero indicates no vegetation cover. For example, bare soil, water bodies, and urban areas have low NDVIs.

[0091] (2) Generation of slope auxiliary data layer

[0092] Slope is the steepness of a certain pixel point on the surface. It is an important quantitative indicator of the spatial characteristics of the terrain in the landslide occurrence area. It is usually expressed in angle or percentage. The occurrence of landslides is closely related to the slope of the terrain. A large amount of public literature shows that the slope of the landslide occurrence area is mostly between 20° and 50°.

[0093] In this embodiment, the digital elevation model data is used to generate the slope auxiliary data layer of the target area.

[0094] The specific steps include:

[0095] Given a pixel (i, j) on the digital elevation model data layer, where i and j are the row and column numbers of the pixel respectively, h(i, j) is used to represent the altitude of the pixel (i, j), in meters;

[0096] Select n×n neighboring pixel areas around pixel (i, j), where n is a positive integer, and calculate the average height change rate of pixel (i, j) in the x direction (horizontal direction):

[0097]

[0098] Among them, cellsize x is the spatial width of a single pixel in the x direction.

[0099] Calculate the average height change rate of pixel (i, j) in the y direction (vertical direction):

[0100]

[0101] Among them, cellsize y is the spatial width of a single pixel in the y direction.

[0102] Calculate the slope of pixel (i,j):

[0103]

[0104] Move the n×n pixel window area to traverse each pixel, repeat the above steps, calculate the slope value of each pixel in the entire target area, and obtain the slope data of the target area, that is, the slope auxiliary data layer B slope ,like Figure 6 In this embodiment, n is preferably, but not limited to, 3.

[0105] Step 4: Vertical association and combination of data layers

[0106] The multispectral reflectance image, digital elevation model data, surface normalized vegetation index and slope data of the target area are vertically correlated and combined according to geographic coordinates to form a data cube that integrates the spatiotemporal spectral characteristic information of landslides and typical background objects.

[0107] This embodiment vertically associates and combines the Sentinel-2 multispectral reflectance images of the target area before and after the earthquake landslide, the calculated surface normalized vegetation index, the digital elevation model data and the slope data according to the geographic coordinates to form a data cube that integrates the spatiotemporal spectral feature information of the landslide and typical background objects, such as Figure 7 shown. Figure 7 The 12 bands of images before and after the disaster, plus two layers of NDVI calculated separately, plus digital elevation model data and slope layers, a total of 28 layers, were fused into a data cube. This data cube contains information on spectral changes before and after the disaster, changes in the surface NDVI, elevation information, and slope information.

[0108] In the data cube, each pixel (coordinate point) contains the multispectral information (spectral characteristics) of the ground objects, the surface normalized vegetation index information, and the corresponding change information (temporal change characteristics) in the remote sensing images before and after the disaster, elevation information and slope information (terrain spatial characteristics), realizing the correlation and fusion of multi-dimensional feature information of ground objects for remote sensing monitoring and identification of landslides.

[0109] Step 5: Labeling of typical ground features

[0110] Through visual interpretation, high-resolution remote sensing image-assisted identification and cross-validation of open source information, some high-quality and high-precision basic sample data of landslides and typical background objects were manually marked.

[0111] In this embodiment, through visual interpretation and high-resolution remote sensing data-assisted identification, combined with the publicly released geological disaster interpretation results of drone images after the Luding earthquake on September 8 and the earthquake emergency investigation results of the Chengdu Mountain Research Institute of the Chinese Academy of Sciences, some high-quality and high-precision landslides and typical background feature samples were manually marked to establish a basic sample data set, such as Figure 8 Among them, there are 4102 marked landslide samples, 4422 vegetation samples, 3848 water samples, and 4966 urban construction area and cloud samples.

[0112] Step 7: Migration and enhancement of typical landform sample data based on region growing algorithm and Mahalanobis distance

[0113] The unlabeled sample points are searched around the labeled sample points by using the region growing algorithm. When the Mahalanobis distance between the labeled sample points and the unlabeled sample points is less than the growth threshold T M When the unlabeled sample points are used as sample points of the same type as the labeled sample points, typical sample data migration enhancement is achieved.

[0114] The specific steps are as follows:

[0115] Step 1: Sequentially scan the labeled samples of various types and take the first labeled basic sample (manually labeled) as the central pixel (i 0 ,j 0 ).

[0116] Step 2: Take the center pixel (i 0 ,j 0 ) as the center, such as Figure 2 As shown, consider the central pixel (i 0 ,j 0 ) in the surrounding 8-connected neighborhoods. If there is no pixel (i, j) marked with the category it belongs to, then if the pixel (i, j) is 0 ,j 0 ) is less than the growth threshold T M , then the growth criterion is satisfied, and the pixel (i, j) and the center pixel (i 0 ,j 0 ) are merged (i.e., they are considered to be similar objects in the same area), typical sample data migration enhancement is achieved, and pixel (i, j) is pushed into the stack (i.e., it meets the conditions and is judged to be the same as the center pixel (i 0 ,j 0 ) belongs to a similar area of ​​land features, and needs to continue growing outward from this pixel point).

[0117] In this step, the calculation formula of Mahalanobis distance is:

[0118]

[0119] Where Y = [y 1 ,y 2 ,",y i ,...,y p ] T , is a set of unlabeled sample points, y i represents the multispectral reflectance, digital elevation model data, surface normalized vegetation index and slope data of the i-th sample point in Y, is a set of labeled sample points, x i represents the multispectral reflectance, digital elevation model data, surface normalized vegetation index and slope data of the i-th sample point in X, c represents the labeled sample category, Σ c is the covariance matrix of the multidimensional features of the c-th class of labeled sample data sets, D M (Y,X c ) can be used to characterize the similarity between labeled sample points and unlabeled sample points.

[0120] Step 3: Take a pixel from the stack and use it as the center pixel (i 0 ,j 0 )Return to step 2.

[0121] Step 4: When the stack is empty or the total number of samples in the stack reaches the upper limit T of a single sample growth num_max When , return to step 1 and take one of the remaining labeled basic samples as the central pixel (i 0 ,j 0 ).

[0122] Step 5: Repeat steps 1-4 until all marked basic sample points are traversed and the growth is completed.

[0123] In this embodiment, based on experimental experience, the growth threshold T M =8, the upper limit of a single sample growth is T num_max =20.

[0124] In terms of similarity measurement, this embodiment uses the traditional Euclidean distance and the Mahalanobis distance selected in this patent for migration enhancement comparison experiments of sample data. The relevant results are as follows: Figure 9a , Figure 9b and Fig.9c As shown. Combined with Figure 3, Figure 8 , Figure 9a , Figure 9b and Fig.9cBy comparing the distribution of various types of ground object samples before and after data enhancement, it can be seen that the introduction of Mahalanobis distance can more accurately characterize the similarity between two sample data points, and the various ground object samples generated after data migration enhancement combined with the regional growing algorithm can maintain a high degree of consistency with the actual distribution of various typical ground objects in remote sensing images.

[0125] To further verify the effectiveness of this embodiment, the classic SVM model and the currently popular CNN model were selected for model training and target recognition experiments. All 17,338 manually labeled basic sample data sets were randomly divided into a training set and a validation set at a ratio of 7:3, wherein the training set was used to train the SVM and CNN models after sample data enhancement according to this embodiment. The kernel function of the SVM model used in the experiment in this embodiment is the radial basis function (RBF) kernel, the regularization parameter C = 20, and the coefficient of the kernel function gamma = 0.1; the basic structure of the CNN model used in the experiment is as follows Fig.10 The target recognition methods involved in this embodiment are all implemented in Python (version: 3.8.12). The experimental platform is configured with an Intel (R) Core (TM) i7 processor, 16G memory, and Windows 10 operating system. The recognition results of various typical ground objects are as follows: Fig.11a , Fig.11b , Fig.11c and Fig.11d The accuracy of various comparison methods is shown in Table 2.

[0126] Table 2 Recognition accuracy of various comparison methods for typical ground objects

[0127]

[0128] Combined with Figure 3, Figure 8 , Fig.11a , Fig.11b , Fig.11c , Fig.11d It can be seen from Table 2 that the method described in this embodiment alleviates the underfitting problem in the training process of the machine learning model by effectively expanding the basic sample data set. The new sample data set obtained after data migration and enhancement processing by the method described in this embodiment is used to train the SVM model and the CNN model, which can effectively improve the recognition accuracy of various models for typical ground objects.

[0129] Embodiment 2

[0130] This embodiment provides a sample data migration and enhancement system, which includes a processing module, a storage module, and a data acquisition module for acquiring multispectral remote sensing images and digital elevation model data of a target area; the data acquisition module is connected to the processing module and sends the multispectral remote sensing image and digital elevation model data to the processing module; the processing module and the storage module are communicatively connected, and the storage module is used to store at least one executable instruction, and the executable instruction enables the processing module to perform operations corresponding to the sample data migration and enhancement method based on the correlation and fusion of spatiotemporal spectral features of remote sensing images based on the multispectral remote sensing image and digital elevation model data, so as to migrate and enhance the sample data.

[0131] Embodiment 3

[0132] This embodiment provides a target recognition method, which obtains samples based on the above-mentioned sample data migration enhancement method based on the correlation fusion of remote sensing image spatiotemporal spectral features; constructs a machine learning model, trains the machine learning model with the migrated and enhanced samples, and uses the trained machine learning model to perform target recognition on the samples to be recognized.

[0133] In this embodiment, the machine learning model is preferably but not limited to the classic SVM model and the currently popular CNN model, etc.

[0134] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0135] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A sample data migration enhancement method based on remote sensing image spatiotemporal feature correlation fusion, characterized in that: The following steps are involved: Obtain multispectral remote sensing images and digital elevation model data of the target area before and after the disaster, pre-process them, and convert the multispectral remote sensing images into multispectral reflectance images; Calculating a surface normalized vegetation index of a target area based on the multispectral reflectance image, and calculating slope data of the target area based on the digital elevation model data; The multispectral reflectance images, digital elevation model data, surface normalized vegetation index and slope data of the target area before and after the disaster are vertically correlated and combined according to geographic coordinates to form a data cube that integrates the spatiotemporal spectral characteristic information of landslides and typical background objects; Marking basic sample data of typical land features in the data cube; Combine the region growing algorithm and Mahalanobis distance algorithm to migrate and enhance the labeled basic sample data of typical features. The steps are as follows: Step 1: Sequentially scan the labeled samples of various types and take the first labeled basic sample as the central pixel (i0, j0); Step 2: Taking the central pixel (i0, j0) as the center, consider the pixels (i, j) in the 8-connected neighborhood around the central pixel (i0, j0) that are not marked with the belonging category. If the Mahalanobis distance between the pixel (i, j) and the central pixel (i0, j0) is less than the growth threshold T M When , the growth criterion is met, and the pixel (i, j) is merged with the central pixel (i0, j0) as the same type of feature in the same area, and the pixel (i, j) is pushed into the stack; Step 3: Take a pixel from the stack and treat it as the center pixel (i0, j0) and return to step 2; Step 4: When the stack is empty or the total number of samples in the stack reaches the upper limit T of a single sample growth num_max When , return to step 1 and take one of the remaining labeled basic samples as the central pixel (i0, j0); Step 5: Repeat steps 1-4 until all marked basic sample points are traversed and the growth is completed.

2. The sample data migration enhancement method based on remote sensing image spatiotemporal feature correlation fusion according to claim 1 is characterized in that: The ratio of the sum of the reflectance image of the near-infrared band and the reflectance image of the red band in the multispectral reflectance image of the target area and the difference between the two is used as the surface normalized vegetation index of the target area.

3. The sample data migration enhancement method based on remote sensing image spatiotemporal feature correlation fusion according to claim 1 is characterized in that: The calculation steps of slope data are: Given a pixel (i, j) on the digital elevation model data, i and j are the row and column numbers of the pixel respectively; Select n×n neighboring pixel areas around pixel (i, j) and calculate the average height change rate of pixel (i, j) in the x direction, where n is a positive integer: Among them, cellsize x is the spatial width of a single pixel in the x direction, and h(i,j) is the altitude of the pixel (i,j); Calculate the average height change rate of pixel (i, j) in the y direction: Among them, cellsize y is the spatial width of a single pixel in the y direction; Calculate the slope of pixel (i,j): The n×n pixel window area is moved to traverse each pixel, and the slope value of each pixel in the entire target area is calculated to obtain the slope data of the target area.

4. The sample data migration and enhancement method based on remote sensing image spatiotemporal feature correlation fusion according to claim 1 is characterized in that: The steps for migrating and enhancing the labeled basic sample data of typical objects are as follows: The unlabeled sample points are searched around the labeled sample points by using the region growing algorithm. When the Mahalanobis distance between the labeled sample points and the unlabeled sample points is less than the growth threshold T M When the unlabeled sample points are used as sample points of the same type as the labeled sample points, typical sample data migration enhancement is achieved.

5. The sample data migration enhancement method based on remote sensing image spatiotemporal feature correlation fusion according to claim 4 is characterized in that: The calculation formula based on Mahalanobis distance is: Where Y = [y1,y2,",y i ,...,y p ] T , is a set of unlabeled sample points, y i represents the multispectral reflectance, digital elevation model data, surface normalized vegetation index and slope data of the i-th sample point in Y, is a set of labeled sample points, x i represents the multispectral reflectance, digital elevation model data, surface normalized vegetation index and slope data of the i-th sample point in X, c represents the labeled sample category, Σ c is the covariance matrix of the multidimensional features of the c-th class of labeled sample data sets, D M (Y,X c ) is used to characterize the similarity between labeled sample points and unlabeled sample points.

6. A sample data migration enhancement system, characterized in that: It includes a processing module, a storage module and a data acquisition module for acquiring multi-spectral remote sensing images and digital elevation model data of a target area; The data acquisition module is connected to the processing module and sends the multispectral remote sensing image and digital elevation model data to the processing module; the processing module is communicatively connected to the storage module, and the storage module is used to store at least one executable instruction, and the executable instruction enables the processing module to perform operations corresponding to the sample data migration enhancement method based on the correlation and fusion of spatiotemporal spectral features of remote sensing images as described in any one of claims 1 to 5 based on the multispectral remote sensing image and digital elevation model data, and perform migration enhancement on the sample data.

7. A target recognition method, characterized in that: Acquire samples based on the sample data migration enhancement method based on remote sensing image spatiotemporal feature correlation fusion as described in any one of claims 1 to 5; Construct a machine learning model, use the samples to train the machine learning model, and use the trained machine learning model to perform target recognition on the samples to be recognized.

Images