A remote sensing image object classification method and system based on convolutional neural network

Abstract

The present invention discloses a remote sensing image object classification method and system based on a convolutional neural network. The method comprises collecting remote sensing images of a preset area and preprocessing the remote sensing images; correcting the remote sensing images using relevant data to obtain a corrected image; and extracting multimodal features from the corrected image to obtain image features; the image features include spectral features, spatial features, texture features, and polarization features; fusing and comparing the image features to obtain comparison data, and constructing a remote sensing image object classification model using the comparison data; optimizing the remote sensing image object classification model based on classification error, inputting the data to be classified into the remote sensing image object classification model, and outputting a classification result. This method not only improves the accuracy of remote sensing image object classification, but also has good interpretability and can be directly applied to remote sensing image object classification systems.

Description

Technical Field

[0001] The present invention relates to the field of image classification, and in particular to a remote sensing image object classification method and system based on convolutional neural network. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing imagery has become an indispensable source of information for fields such as Earth observation, environmental monitoring, resource surveys, and disaster warning. With the continuous advancement of sensor technology, the resolution of remote sensing imagery continues to increase, and the information it contains is becoming richer and more complex. This provides more possibilities for object classification, but also brings unprecedented challenges. Traditional remote sensing imagery object classification methods, such as pixel-based classification and object-oriented methods, while able to meet application requirements to a certain extent, often suffer from low classification accuracy, weak generalization ability, and insufficient ability to analyze complex scenes when processing complex remote sensing data such as high-resolution, multispectral, multitemporal, and polarimetric data.

[0003] However, directly applying CNNs to remote sensing imagery for object classification still faces several challenges. First, remote sensing images are affected by multiple factors, such as atmospheric conditions, shooting angles, and seasonal changes, and exhibit significant spatiotemporal variability, which requires the classification model to be highly robust. Second, the types of objects in remote sensing images are diverse, and often exhibit "same object, different spectra" or "different objects, same spectra" phenomena. A single feature extraction method cannot fully describe the characteristics of the objects. Finally, remote sensing image data is large in volume and the annotation cost is high. Therefore, Kou Dai invented a new object classification method to improve the accuracy and precision of object classification. Summary of the Invention

[0004] The purpose of this invention is to provide a remote sensing image object classification method based on convolutional neural network.

[0005] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0006] The present invention comprises the following steps:

[0007] Collecting remote sensing images of a preset area and preprocessing the remote sensing images;

[0008] Correcting the remote sensing image based on the relevant data to obtain a corrected image, and performing multimodal feature extraction on the corrected image to obtain image features; the image features include spectral features, spatial features, texture features, and polarization features;

[0009] Performing fusion comparison on the image features to obtain comparison data, and using the comparison data to construct a remote sensing image object classification model;

[0010] The remote sensing image ground object classification model is optimized according to the classification error, the data to be classified is input into the remote sensing image ground object classification model, and the classification result is output.

[0011] Furthermore, the method of correcting the remote sensing image based on the relevant data to obtain a corrected image includes:

[0012] The feature information of remote sensing images is obtained through global average pooling, and the expression is:

[0013]

[0014] The xth remote sensing image is k x , remote sensing image k x The characteristic information is The height is U, the width is S, and the remote sensing image k x The modulus is |k x |, the number of remote sensing images is N x , remote sensing image k x The clarity is ε x ;

[0015] According to the spatial information of remote sensing images, the similarity of remote sensing images is calculated:

[0016]

[0017] The cth remote sensing image is k c , similarity tolerance parameter is E, calibration error is κ, remote sensing image k x and remote sensing images c The similarity is d(k x ,k c );

[0018] Remote sensing images with similarity greater than 0.526 are corrected according to their clarity, and the local remote sensing images with the highest clarity are used to replace the local remote sensing images with low clarity, and the replaced images are combined and output as the corrected image.

[0019] Furthermore, the method of performing multimodal feature extraction on the corrected image to obtain image features includes:

[0020] The corrected image is input into the multimodal feature extraction model, the information embedding transformation is performed on the corrected image, and the generated feature maps are spliced. The expression is:

[0021] g=δ(F1[d h ,d w ])

[0022]

[0023] The concatenated feature map is g, and the a-th channel output of width w is The output of the ath channel of width h is The height is h, the width is W, and the coordinates of the corrected image with length h and height i in the ath channel are u a (h,i), the corrected image coordinates of the ath channel with abscissa w and ordinate j are u a (w, j), the activation function is δ(·), and the first convolution function is F1[·,·];

[0024] The concatenated feature map is segmented along the spatial dimension to obtain the feature map, and the feature map is increased in dimension by convolution. The attention vector is obtained by combining the activation function, which is expressed as:

[0025] y a (i,j)=u a (i,j)×σ(F h (g h ))×σ(F w (g w ))

[0026] The logistic activation function is σ(·), and the convolution function of width w is F w , the convolution function of width h is F h , the attention vector of width w is g w , the attention vector of width h is g h , the corrected image coordinates of the ath channel with abscissa i and ordinate j are u a (i, j), the output coordinate of the ath channel with horizontal coordinate i and vertical coordinate j is y a (i,j);

[0027] The attention vector is weighted and the expression is:

[0028]

[0029] The transpose is T, the height is H, and the relative encoding position of the horizontal coordinate i and the vertical coordinate j is The logistic activation function at the horizontal coordinate i and vertical coordinate j is ij The weight is φ ij , the element with horizontal coordinate i and vertical coordinate j is θ ij , the i-th corrected image is u i , the j-th key vector is The i-th weighted attention vector is R i ;

[0030] The weighted attention vector is output as image features.

[0031] Furthermore, the method of fusing and comparing the image features to obtain comparison data includes:

[0032] Clustering is used to reduce the dimension of image features, and the expression is:

[0033]

[0034] The center value of the sth dimension of the bth cluster is f b (s), the value of the s-th dimension of the p-th image feature is g p (s), the p-th image feature is g p , the bth cluster is l b , image feature g p Belongs to cluster l b The distance is l b (g p ), the dimension reduction feature of the b-th cluster s-th dimension is C bs , transposed to T, the bias of the b-th cluster is K b , the weight vector of the y-th cluster is The error constant is τ;

[0035] Calculate the similarity between dimensionality reduction features:

[0036]

[0037] The vth image feature is g v , image feature g p and image features g v The similarity is The suppression factor is ρ and the Euclidean norm is ‖·‖2;

[0038] Clustering is performed based on similarity, and the two dimensionality reduction features with the greatest similarity are grouped into one category to obtain the classification;

[0039] Calculate the weighting coefficients for the categories:

[0040]

[0041] The weighted coefficient of the pth image feature in the bth cluster is The adjustable coefficient is γ, and the image feature g p The probability of occurrence is P(g p ), the number of image features of cluster b is M b ;

[0042] Image feature fusion is performed according to the weighted coefficient to obtain the fusion feature, which is expressed as:

[0043]

[0044] The fusion feature of the b-th cluster is χ b , the control coefficient is λ.

[0045] Furthermore, a method for constructing a remote sensing image object classification model using the comparison data includes:

[0046] Remote sensing image object classification models include random forest algorithms, deep learning feature matching algorithms, and machine learning algorithms;

[0047] The random forest algorithm randomly divides the comparison data into training data and test data in a ratio of 7:2;

[0048] The deep learning feature matching algorithm learns the representation and matching rules of ground object features by training neural networks, realizes automatic detection and accurate matching of training data, and obtains the matching degree;

[0049] Machine learning algorithms automatically classify new remote sensing images by learning the matching degree between features and categories of objects from training data;

[0050] The test data is input into the remote sensing image object classification model to obtain the predicted classification data. According to the variance of the predicted classification data and the actual classification data, the learning rate of the remote sensing image object classification model is adjusted according to the variance until the variance reaches the minimum value.

[0051] Furthermore, the method for optimizing the remote sensing image object classification model according to the classification error includes:

[0052] The particle swarm optimization algorithm is introduced, and the classification error is used as the fitness function to calculate the particle fitness, and the position of the particle with the smallest fitness is taken as the optimal position;

[0053] Calculate the initial position of the particle:

[0054]

[0055] The initial position of the zth particle in the tth iteration is The random number between 0 and 1 is r1, and the upper bound of the population space range is V u , the lower bound of the population space range is V l ;

[0056] Update the particle's position. The expression is:

[0057]

[0058] The updated position of the zth particle in the t+1th iteration is The normal distribution random number is β, the random number from 0 to 1 is r2, and the position of the random particle at the tth iteration is

[0059] Introducing inertia weight, the expression is:

[0060]

[0061] The inertia weight of the t-th iteration is μ(t), and the maximum value of the inertia weight is μ max , the maximum number of iterations is t max , the current iteration number is t, and the random number between 0 and 1 is r3;

[0062] Update the particle position according to the inertia weight to obtain the inertial position. The expression is:

[0063]

[0064] The inertial position of the zth particle in the t+1th iteration is The updated position of the zth particle at the tth iteration is The random number between 0 and 1 is r4, and the optimal position of the particle in the tth iteration is

[0065] Update the particle position according to the Levy flight to obtain the flight position, the expression is:

[0066]

[0067] The flight position of the wth particle in the t+1th iteration is The Levy flight step length is l, and the updated position of the zth particle in the tth iteration is

[0068] Iterate continuously until the maximum number of iterations is reached, otherwise update the inertia weight.

[0069] The second aspect is a remote sensing image object classification system based on convolutional neural network, comprising:

[0070] Data acquisition module: used to collect remote sensing images of a preset area and pre-process the remote sensing images;

[0071] Correction and extraction module: used for correcting the remote sensing image based on the relevant data to obtain a corrected image, and performing multimodal feature extraction on the corrected image to obtain image features; the image features include spectral features, spatial features, texture features and polarization features;

[0072] Fusion construction module: used for fusing and comparing the image features to obtain comparison data, and using the comparison data to construct a remote sensing image object classification model;

[0073] Optimization output module: used to optimize the remote sensing image object classification model according to the classification error, input the data to be classified into the remote sensing image object classification model, and output the classification result.

[0074] In a second aspect, an embodiment of the present application further provides an electronic device, including:

[0075] A processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform the method steps described in the first aspect.

[0076] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method steps described in the first aspect.

[0077] The beneficial effects of the present invention are:

[0078] The present invention is a remote sensing image object classification method and system based on convolutional neural network. Compared with the existing technology, the present invention has the following technical effects:

[0079] The present invention can improve the accuracy of remote sensing image ground object classification based on convolutional neural networks through preprocessing, correction processing, multimodal feature extraction, fusion comparison, model construction and model optimization steps, thereby improving the precision of remote sensing image ground object classification based on convolutional neural networks. Optimizing remote sensing image ground object classification based on convolutional neural networks can greatly save resources and improve work efficiency. It can realize automatic ground object classification of remote sensing images, and perform image correction and fusion comparison on remote sensing image ground object classification based on convolutional neural networks in real time. It is of great significance to remote sensing image ground object classification based on convolutional neural networks, can adapt to remote sensing image ground object classification of different standards and different remote sensing image ground object classification requirements, and has a certain universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a flow chart of the steps of a remote sensing image object classification method based on a convolutional neural network of the present invention;

[0081] Figure 2 This is a schematic diagram of the structure of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0082] The present invention will be further described below through specific examples. The illustrative examples and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0083] The present invention provides a remote sensing image object classification method and system based on convolutional neural network, comprising the following steps:

[0084] like Figure 1 As shown, in this embodiment, the following steps are included:

[0085] Collecting remote sensing images of a preset area and preprocessing the remote sensing images;

[0086] In the actual assessment, XXX Farm was selected as the research object and remote sensing images of the farm in May were extracted. The farm included the main crop planting areas of wheat, corn, rice and cotton.

[0087] Preprocessing includes radiometric calibration, geometric correction and denoising;

[0088] Correcting the remote sensing image based on the relevant data to obtain a corrected image, and performing multimodal feature extraction on the corrected image to obtain image features; the image features include spectral features, spatial features, texture features, and polarization features;

[0089] In the actual evaluation, the remote sensing images were corrected by combining historical crop planting records, farmland soil maps and meteorological data. The spectral features in the image features were used to extract the spectra of the crop growth stages, the spatial features were used to obtain the shape, size and arrangement pattern of the crop fields, the texture features were used to extract the texture of the crop canopy, and the polarization features were used to extract the scattering characteristics of the crop canopy.

[0090] Performing fusion comparison on the image features to obtain comparison data, and using the comparison data to construct a remote sensing image object classification model;

[0091] The remote sensing image ground object classification model is optimized according to the classification error, the data to be classified is input into the remote sensing image ground object classification model, and the classification result is output.

[0092] In this embodiment, the method of correcting the remote sensing image based on the relevant data to obtain a corrected image includes:

[0093] The feature information of remote sensing images is obtained through global average pooling, and the expression is:

[0094]

[0095] The xth remote sensing image is k x , remote sensing image k x The characteristic information is The height is U, the width is S, and the remote sensing image k x The modulus is |k x |, the number of remote sensing images is N x , remote sensing image k x The clarity is ε x ;

[0096] According to the spatial information of remote sensing images, the similarity of remote sensing images is calculated:

[0097]

[0098] The cth remote sensing image is k c , similarity tolerance parameter is E, calibration error is κ, remote sensing image k x and remote sensing images c The similarity is d(k x ,k c );

[0099] Remote sensing images with similarity greater than 0.526 are corrected according to their clarity, and the local remote sensing images with the highest clarity are used to replace the local remote sensing images with low clarity, and the replaced images are combined and output as the corrected image.

[0100] In this embodiment, the method for performing multimodal feature extraction on the corrected image to obtain image features includes:

[0101] The corrected image is input into the multimodal feature extraction model, the information embedding transformation is performed on the corrected image, and the generated feature maps are spliced. The expression is:

[0102] g=δ(F1[d h ,d w ])

[0103]

[0104] The concatenated feature map is g, and the a-th channel output of width w is The output of the ath channel of width h is The height is h, the width is W, and the coordinates of the corrected image with length h and height i in the ath channel are u a (h,i), the corrected image coordinates of the ath channel with abscissa w and ordinate j are u a (w, j), the activation function is δ(·), and the first convolution function is F1[·,·];

[0105] The concatenated feature map is segmented along the spatial dimension to obtain the feature map, and the feature map is increased in dimension by convolution. The attention vector is obtained by combining the activation function, which is expressed as:

[0106] y a (i,j)=u a (i,j)×σ(F h (g h ))×σ(F w (g w ))

[0107] The logistic activation function is σ(·), and the convolution function of width w is F w , the convolution function of width h is F h , the attention vector of width w is g w , the attention vector of width h is g h, the corrected image coordinates of the ath channel with abscissa i and ordinate j are u a (i, j), the output coordinate of the ath channel with horizontal coordinate i and vertical coordinate j is y a (i,j);

[0108] The attention vector is weighted and the expression is:

[0109]

[0110] The transpose is T, the height is H, and the relative encoding position of the horizontal coordinate i and the vertical coordinate j is The logistic activation function at the horizontal coordinate i and vertical coordinate j is ij The weight is φ ij , the element with horizontal coordinate i and vertical coordinate j is θ ij , the i-th corrected image is u i , the j-th key vector is The i-th weighted attention vector is R i ;

[0111] The weighted attention vector is output as image features.

[0112] In this embodiment, the method of fusing and comparing the image features to obtain comparison data includes:

[0113] Clustering is used to reduce the dimension of image features, and the expression is:

[0114]

[0115]

[0116] The center value of the sth dimension of the bth cluster is f b (s), the value of the s-th dimension of the p-th image feature is g p (s), the p-th image feature is g p , the bth cluster is l b , image feature g p Belongs to cluster l b The distance is l b (g p ), the dimension reduction feature of the b-th cluster s-th dimension is C bs , transposed to T, the bias of the b-th cluster is K b , the weight vector of the y-th cluster is The error constant is τ;

[0117] Calculate the similarity between dimensionality reduction features:

[0118]

[0119] The vth image feature is g v , image feature g p and image features g v The similarity is The suppression factor is ρ and the Euclidean norm is ‖·‖2;

[0120] Clustering is performed based on similarity, and the two dimensionality reduction features with the greatest similarity are grouped into one category to obtain the classification;

[0121] Calculate the weighting coefficients for the categories:

[0122]

[0123] The weighted coefficient of the pth image feature in the bth cluster is The adjustable coefficient is γ, and the image feature g p The probability of occurrence is P(g p ), the number of image features of cluster b is M b ;

[0124] Image feature fusion is performed according to the weighted coefficient to obtain the fusion feature, which is expressed as:

[0125]

[0126] The fusion feature of the b-th cluster is χ b , the control coefficient is λ.

[0127] In this embodiment, the method of constructing a remote sensing image object classification model using the comparison data includes:

[0128] Remote sensing image object classification models include random forest algorithms, deep learning feature matching algorithms, and machine learning algorithms;

[0129] The random forest algorithm randomly divides the comparison data into training data and test data in a ratio of 7:2;

[0130] The deep learning feature matching algorithm learns the representation and matching rules of ground object features by training neural networks, realizes automatic detection and accurate matching of training data, and obtains the matching degree;

[0131] Machine learning algorithms automatically classify new remote sensing images by learning the matching degree between features and categories of objects from training data;

[0132] The test data is input into the remote sensing image object classification model to obtain the predicted classification data. According to the variance of the predicted classification data and the actual classification data, the learning rate of the remote sensing image object classification model is adjusted according to the variance until the variance reaches the minimum value.

[0133] In this embodiment, the method for optimizing the remote sensing image object classification model according to the classification error includes:

[0134] The particle swarm optimization algorithm is introduced, and the classification error is used as the fitness function to calculate the particle fitness, and the position of the particle with the smallest fitness is taken as the optimal position;

[0135] Calculate the initial position of the particle:

[0136]

[0137] The initial position of the zth particle in the tth iteration is The random number between 0 and 1 is r1, and the upper bound of the population space range is V u , the lower bound of the population space range is V l ;

[0138] Update the particle's position. The expression is:

[0139]

[0140] The updated position of the zth particle in the t+1th iteration is The normal distribution random number is β, the random number from 0 to 1 is r2, and the position of the random particle at the tth iteration is

[0141] Introducing inertia weight, the expression is:

[0142]

[0143] The inertia weight of the t-th iteration is μ(t), and the maximum value of the inertia weight is μ max , the maximum number of iterations is t max , the current iteration number is t, and the random number between 0 and 1 is r3;

[0144] Update the particle position according to the inertia weight to obtain the inertial position. The expression is:

[0145]

[0146] The inertial position of the zth particle in the t+1th iteration is The updated position of the zth particle at the tth iteration is The random number between 0 and 1 is r4, and the optimal position of the particle in the tth iteration is

[0147] Update the particle position according to the Levy flight to obtain the flight position, the expression is:

[0148]

[0149] The flight position of the wth particle in the t+1th iteration is The Levy flight step length is l, and the updated position of the zth particle in the tth iteration is

[0150] Iterate continuously until the maximum number of iterations is reached, otherwise update the inertia weight.

[0151] The second aspect is a remote sensing image object classification system based on convolutional neural network, comprising:

[0152] Data acquisition module: used to collect remote sensing images of a preset area and pre-process the remote sensing images;

[0153] Correction and extraction module: used for correcting the remote sensing image based on the relevant data to obtain a corrected image, and performing multimodal feature extraction on the corrected image to obtain image features; the image features include spectral features, spatial features, texture features and polarization features;

[0154] Fusion construction module: used for fusing and comparing the image features to obtain comparison data, and using the comparison data to construct a remote sensing image object classification model;

[0155] Optimization output module: used to optimize the remote sensing image object classification model according to the classification error, input the data to be classified into the remote sensing image object classification model, and output the classification result.

[0156] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 2 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0157] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0158] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0159] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, logically forming a remote sensing image object classification device based on a convolutional neural network. The processor executes the program stored in the memory and is specifically configured to perform any of the aforementioned remote sensing image object classification methods based on a convolutional neural network.

[0160] The above application Figure 1 The embodiment disclosed herein discloses a method for classifying objects in remote sensing images based on a convolutional neural network, which can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or by software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0161] The electronic device may also perform Figure 1 A remote sensing image object classification method based on convolutional neural network is proposed and implemented Figure 1The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0162] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple application programs, any one of the aforementioned remote sensing image object classification methods based on convolutional neural networks is executed.

[0163] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0164] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0165] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0166] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0167] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0168] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0169] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0170] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0171] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0172] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A remote sensing image object classification method based on convolutional neural network, characterized in that: The following steps are involved: Collecting remote sensing images of a preset area and preprocessing the remote sensing images; Correcting the remote sensing image based on relevant data to obtain a corrected image, and performing multimodal feature extraction on the corrected image to obtain image features; the image features include spectral features, spatial features, texture features, and polarization features; Performing fusion comparison on the image features to obtain comparison data, and using the comparison data to construct a remote sensing image object classification model; including: Clustering is used to reduce the dimension of image features, and the expression is: The center value of the sth dimension of the bth cluster is f b (s), the value of the s-th dimension of the p-th image feature is g p (s), the p-th image feature is g p , the bth cluster is Image features g p Belong to cluster The distance is The dimensionality reduction feature of the sth dimension of the bth cluster is C bs , transposed to T, the bias of the b-th cluster is K b , the weight vector of the y-th cluster is The error constant is τ; Calculate the similarity between dimensionality reduction features: The vth image feature is g v , image feature g p and image features g v The similarity is The suppression factor is ρ and the Euclidean norm is ||·||2; Clustering is performed based on similarity, and the two dimensionality reduction features with the greatest similarity are grouped into one category to obtain the classification; Calculate the weighting coefficients for the categories: The weighted coefficient of the pth image feature in the bth cluster is The adjustable coefficient is γ, and the image feature g p The probability of occurrence is P(g p ), the number of image features of cluster b is M b ; Image feature fusion is performed according to the weighted coefficient to obtain the fusion feature, which is expressed as: The fusion feature of the b-th cluster is χ b , the control coefficient is λ; The remote sensing image ground object classification model is optimized according to the classification error, the data to be classified is input into the remote sensing image ground object classification model, and the classification result is output.

2. The remote sensing image object classification method based on convolutional neural network according to claim 1, characterized in that: The method of correcting the remote sensing image to obtain a corrected image by correcting relevant data includes: The feature information of remote sensing images is obtained through global average pooling, and the expression is: The xth remote sensing image is k x , remote sensing image k x The characteristic information is The height is U, the width is S, and the remote sensing image k x The modulus is |k x |, the number of remote sensing images is N x , remote sensing image k x The clarity is ε x ; According to the spatial information of remote sensing images, the similarity of remote sensing images is calculated: The cth remote sensing image is k c , similarity tolerance parameter is E, calibration error is κ, remote sensing image k x and remote sensing images k c The similarity is Remote sensing images with similarity greater than 0.526 are corrected according to their clarity, and the local remote sensing images with the highest clarity are used to replace the local remote sensing images with low clarity, and the replaced images are combined and output as the corrected image.

3. The remote sensing image object classification method based on convolutional neural network according to claim 1, characterized in that: The method for performing multimodal feature extraction on the corrected image to obtain image features includes: The corrected image is input into the multimodal feature extraction model, the information embedding transformation is performed on the corrected image, and the generated feature maps are spliced. The expression is: g=δ(F1[d h ,d w ]) The concatenated feature map is g, and the a-th channel output of width w is The output of the ath channel of width h is The height is h, the width is W, and the coordinates of the corrected image with length h and height i in the ath channel are u a (h,i), the corrected image coordinates of the ath channel with abscissa w and ordinate j are u a (w, j), the activation function is δ(·), and the first convolution function is F1[·,·]; The concatenated feature map is segmented along the spatial dimension to obtain the feature map, and the feature map is increased in dimension by convolution. The attention vector is obtained by combining the activation function, which is expressed as: y a (i,j)=u a (i,j)×σ(F h (g h ))×σ(F w (g w )) The logistic activation function is σ(·), and the convolution function of width w is F w , the convolution function of width h is F h , the attention vector of width w is g w , the attention vector of width h is g h , the corrected image coordinates of the ath channel with abscissa i and ordinate j are u a (i, j), the output coordinate of the ath channel with horizontal coordinate i and vertical coordinate j is y a (i,j); The attention vector is weighted and the expression is: The transpose is T, the height is H, and the relative encoding position of the horizontal coordinate i and the vertical coordinate j is The logistic activation function at the horizontal coordinate i and vertical coordinate j is ij The weight is φ ij , the element of horizontal coordinate i and vertical coordinate j is θ ij , the i-th corrected image is u i , the j-th key vector is The i-th weighted attention vector is R i ; The weighted attention vector is output as image features.

4. The remote sensing image object classification method based on convolutional neural network according to claim 1, characterized in that: The method for constructing a remote sensing image object classification model using the comparison data includes: Remote sensing image object classification models include random forest algorithms, deep learning feature matching algorithms, and machine learning algorithms; The random forest algorithm randomly divides the comparison data into training data and test data in a ratio of 7:2; The deep learning feature matching algorithm learns the representation and matching rules of ground object features by training neural networks, realizes automatic detection and accurate matching of training data, and obtains the matching degree; Machine learning algorithms automatically classify new remote sensing images by learning the matching degree between features and categories of objects from training data; The test data is input into the remote sensing image object classification model to obtain the predicted classification data. According to the variance of the predicted classification data and the actual classification data, the learning rate of the remote sensing image object classification model is adjusted according to the variance until the variance reaches the minimum value.

5. The remote sensing image object classification method based on convolutional neural network according to claim 1, characterized in that: The method for optimizing the remote sensing image ground object classification model according to the classification error includes: The particle swarm optimization algorithm is introduced, and the classification error is used as the fitness function to calculate the particle fitness, and the position of the particle with the smallest fitness is taken as the optimal position; Calculate the initial position of the particle: The initial position of the zth particle in the tth iteration is The random number between 0 and 1 is r1, and the upper bound of the population space range is V u , the lower bound of the population space range is V l ; Update the particle's position. The expression is: The updated position of the zth particle in the t+1th iteration is The normal distribution random number is β, the random number from 0 to 1 is r2, and the position of the random particle at the tth iteration is Introducing inertia weight, the expression is: The inertia weight of the t-th iteration is μ(t), and the maximum value of the inertia weight is μ max , the maximum number of iterations is t max , the current iteration number is t, and the random number between 0 and 1 is r3; Update the particle position according to the inertia weight to obtain the inertial position. The expression is: The inertial position of the zth particle in the t+1th iteration is The updated position of the zth particle at the tth iteration is The random number between 0 and 1 is r4, and the optimal position of the particle in the tth iteration is Update the particle position according to the Levy flight to obtain the flight position, the expression is: The flight position of the wth particle in the t+1th iteration is Levy's flight step length is The updated position of the zth particle at the tth iteration is Iterate continuously until the maximum number of iterations is reached, otherwise update the inertia weight.

6. A remote sensing image object classification system based on convolutional neural network, used to execute the method according to any one of claims 1 to 5, characterized in that: include: Data acquisition module: used to collect remote sensing images of a preset area and pre-process the remote sensing images; Correction and extraction module: used to correct the remote sensing image based on relevant data to obtain a corrected image, and perform multimodal feature extraction on the corrected image to obtain image features; the image features include spectral features, spatial features, texture features, and polarization features; Fusion construction module: used for fusing and comparing the image features to obtain comparison data, and using the comparison data to construct a remote sensing image object classification model; Optimization output module: used to optimize the remote sensing image object classification model according to the classification error, input the data to be classified into the remote sensing image object classification model, and output the classification result.

7. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing one or more programs, wherein when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the method according to any one of claims 1 to 5.

Images