Remote sensing image semantic segmentation method and device based on hyperbolic neural network

Abstract

The invention discloses a remote sensing image semantic segmentation method and device based on a hyperbolic neural network, and the method comprises the steps: constructing a hyperbolic space neural network model based on an encoder-decoder structure, obtaining a plurality of earth observation remote sensing images as a training set, and carrying out the training of the neural network model, and the semantic segmentation model is used for carrying out accurate semantic segmentation prediction on the earth observation remote sensing image. According to the neural network, a convolutional layer, batch normalization, nonlinear activation, residual connection, a Softmax function and a deconvolutional layer are mapped to a hyperbolic space by adopting a Lorentz model so as to learn low-distortion remote sensing image representation; pairwise correlation in a hyperbolic space and a Euclidean space is simulated at the same time in a parallel mode, and hyperspherical context dependency and Euclidean affinity are effectively combined. According to the method, the hierarchical structure and the spatial relationship in the remote sensing image can be more effectively captured, so that more accurate semantic segmentation is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image processing, and in particular relates to a remote sensing image semantic segmentation method and device based on a hyperbolic neural network. Background Art

[0002] Semantic segmentation is a key task in computer vision, which involves labeling each pixel in an image for classification at the pixel level. Unlike traditional image prediction, semantic segmentation provides a detailed map of objects, capturing their precise location and shape. As segmentation techniques continue to advance, they are increasingly applied to the complex, data-intensive challenge of remote sensing. Remote sensing images contain detailed information about ground objects and often face problems such as imbalanced class distribution. In addition, the variability in shape, color, and texture of ground objects leads to significant intra-class differences and tiny inter-class differences. These factors pose significant challenges to the semantic segmentation of remote sensing images.

[0003] Many researchers have conducted extensive research in the field of semantic segmentation of remote sensing images. Initially, semantic segmentation of remote sensing images relied on methods that used manually designed features. However, with the increase in the resolution of remote sensing images, traditional methods have difficulty accurately extracting features and achieving precise segmentation. These methods usually require a lot of domain-specific knowledge and technology, resulting in poor adaptability and increased development costs.

[0004] In recent years, the booming development of deep learning, especially convolutional neural networks, has greatly pushed the performance boundaries of tasks such as image segmentation, and fully demonstrated the flexibility and efficiency of such models in feature extraction and pattern recognition. However, traditional convolutional neural networks are limited by their fixed geometric structure and limited local receptive field, making it difficult to fully capture long-distance contextual information in images, which is crucial for improving the distinguishability of feature representations. To make up for this shortcoming, researchers have explored two main strategies to enhance the integration of contextual information. First, by introducing multi-scale dilated convolution or pooling mechanisms, it aims to capture and fuse feature information at different scales, thereby enriching the model's understanding of the scene. Second, it integrates attention modules that can dynamically identify and focus on key areas in the image, effectively capture long-distance dependencies, and further enhance the model's context perception capabilities.

[0005] Despite this, most current mainstream models focus on data processing in Euclidean space, and their original design is to process data with regular grid structures. However, many data in the real world, such as social networks and knowledge graphs, present highly complex and irregular non-Euclidean structures, which makes it difficult for models to accurately capture and express their inherent geometric and topological characteristics when processing such data. Summary of the invention

[0006] The purpose of the present invention is to provide a method and device for semantic segmentation of remote sensing images based on a hyperbolic neural network. The method maps features to hyperbolic space and Euclidean space, utilizes the characteristics of hyperbolic space, can more naturally represent hierarchical structures and spatial relationships, effectively integrates contextual dependencies, and achieves the purpose of accurate extraction of remote sensing image fine semantic segmentation targets.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] In one aspect, the present invention provides a remote sensing image semantic segmentation method based on a hyperbolic neural network, comprising:

[0009] Acquire a target remote sensing image, wherein the target remote sensing image includes a target object of interest and background objects;

[0010] Inputting the acquired target remote sensing image data into a pre-trained remote sensing image semantic segmentation model based on a hyperbolic neural network, and outputting a prediction map containing semantic information, i.e., a semantic segmentation result; in the prediction map, different target objects or regions are divided and labeled into different semantic categories;

[0011] The remote sensing image semantic segmentation model based on the hyperbolic neural network includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; the hyperbolic encoding block and the hyperbolic decoding block process and fuse the input features in the hyperbolic space and the Euclidean space, and enhance the hierarchical structure and spatial relationship of the feature map.

[0012] Preferably, the training process of the remote sensing image semantic segmentation model based on the hyperbolic neural network is as follows:

[0013] Obtaining an earth observation remote sensing image data set, and randomly dividing it into a training set and a test set according to a ratio; the earth observation remote sensing image contains a target object of interest and a background object;

[0014] Constructing a hyperbolic space neural network model based on an encoder-decoder structure, wherein the hyperbolic space neural network model includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks;

[0015] The training set images are input into the hyperbolic space neural network model for training. During the training process, the model parameters are optimized using the hyperbolic normalized cross entropy loss function so that the model output gradually approaches the true label, and finally the model converges to obtain the network model with the optimal parameters as the trained remote sensing image semantic segmentation model;

[0016] The test set is used to verify the performance of the trained remote sensing image semantic segmentation model.

[0017] Preferably, the hyperbolic encoder includes five hyperbolic encoding blocks, and the five hyperbolic encoding blocks have the same structure and the same data processing method;

[0018] The hyperbolic decoder includes five hyperbolic decoding blocks, and the five hyperbolic decoding blocks have the same structure and the same data processing method.

[0019] Preferably, inputting the training set images into the hyperbolic space neural network model for training comprises:

[0020] The input original earth observation remote sensing image is mapped to the hyperbolic space through the logarithmic mapping function to obtain the input hyperbolic feature ;

[0021] The hyperbolic feature Input into the hyperbolic encoder, and each hyperbolic encoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic encoding features , ;

[0022] The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder, and each hyperbolic decoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic decoding features , ;

[0023] Hyperbolic decoding features output by the first hyperbolic decoding block using Lorentz convolution coupled with a Lorentz normalized exponential function Assign a semantic label to each pixel as the output result of semantic segmentation.

[0024] Preferably, the hyperbolic feature The data is input into the hyperbolic encoder, and the data processing process of each hyperbolic encoding block in the hyperbolic encoder is as follows:

[0025] The hyperbolic features of the input in the hyperbolic space Perform convolution operation and use Lorentz convolution layer to extract local features ;

[0026] For the local features Perform Lorentz batch normalization to obtain features ;

[0027] After normalization, the features Use Lorentz nonlinear activation function to perform nonlinear activation to obtain features ;

[0028] The characteristics Input into the Euclidean similarity enhanced hyperbolic attention module to obtain the feature ;

[0029] The characteristics and the characteristics Obtaining features using Lorentz residual connections ;

[0030] The characteristics The above method is used again to perform Lorentz convolution, Lorentz batch normalization, nonlinear activation, Euclidean similarity enhanced hyperbolic attention module processing and Lorentz residual connection to obtain the features ;

[0031] Lorentz average pooling is used to Downsample to get the output of the hyperbolic encoding block , .

[0032] Preferably, the Euclidean similarity enhanced hyperbolic attention module has the following data processing process:

[0033] The characteristics Input to both the hyperbolic branch and the Euclidean branch;

[0034] In hyperbolic space, for the feature After performing Lorentz convolution, batch normalization, and nonlinear activation operations in sequence, the inverse Lorentz distance between features is calculated, and the inverse Lorentz distance is normalized using Lorentz Softmax to generate a hyperbolic attention map. ;

[0035] The characteristics Use logarithmic mapping to map from hyperbolic space to Euclidean space, use standard convolution layer (Conv) to convolve the mapped features, and use standard Softmax function to normalize the convolved feature map to generate Euclidean attention map ;

[0036] The hyperbolic attention map and the Euclidean attention map Through weighted summation, we get the Euclidean similarity enhanced hyperbolic attention map ;

[0037] The characteristics and the Euclidean similarity enhanced hyperbolic attention graph After feature aggregation, residual connection is used to obtain the output features of the Euclidean similarity enhanced hyperbolic attention module .

[0038] Preferably, in the hyperbolic encoder, the input of the first hyperbolic encoding block is the hyperbolic feature , No. The input of the hyperbolic encoding block is The output of the hyperbolic encoding block, .

[0039] Preferably, the hyperbolic encoding feature output by the fifth hyperbolic encoding block is Input into the hyperbolic decoder, the data processing process of each hyperbolic decoding block in the hyperbolic decoder is the same as the data processing process of the hyperbolic encoding block;

[0040] In the hyperbolic decoder, the input of the fifth hyperbolic decoding block is the hyperbolic encoding feature , No. The input of the hyperbolic decoding block is The output of the hyperbolic decoding block, .

[0041] In a second aspect, the present invention provides a remote sensing image semantic segmentation device based on a hyperbolic neural network, which is used to implement the above-mentioned remote sensing image semantic segmentation method based on a hyperbolic neural network, and the device comprises:

[0042] A data acquisition module is used to acquire a target remote sensing image, wherein the target remote sensing image includes a target object of interest and a background object;

[0043] A semantic output module is used to input the acquired target remote sensing image data into a pre-trained remote sensing image semantic segmentation model based on a hyperbolic neural network, and output a prediction map containing semantic information, i.e., a semantic segmentation result; in the prediction map, different target objects or regions are divided and labeled into different semantic categories;

[0044] The remote sensing image semantic segmentation model based on the hyperbolic neural network includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; the hyperbolic encoding block and the hyperbolic decoding block process and fuse the input features in the hyperbolic space and the Euclidean space, and enhance the hierarchical structure and spatial relationship of the feature map.

[0045] Preferably, the device further comprises a model training module,

[0046] The model training module is specifically used to:

[0047] Obtaining an earth observation remote sensing image data set, and randomly dividing it into a training set and a test set according to a ratio; the earth observation remote sensing image contains a target object of interest and a background object;

[0048] Constructing a hyperbolic space neural network model based on an encoder-decoder structure, wherein the hyperbolic space neural network model includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks;

[0049] The training set image is input into the hyperbolic space neural network model. In the hyperbolic space neural network model, the input original earth observation remote sensing image is mapped to the hyperbolic space through a logarithmic mapping function to obtain the input hyperbolic feature ; The hyperbolic feature Input into the hyperbolic encoder, and each hyperbolic encoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic encoding features , ; The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder, and each hyperbolic decoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic decoding features , ; Use Lorentz convolution coupled with Lorentz normalized exponential function as the hyperbolic decoding feature output of the first hyperbolic decoding block Assign a semantic label to each pixel; compare the model output with the true label, and use the hyperbolic normalized cross entropy loss function to optimize the model parameters so that the model output gradually approaches the true label, and finally reach model convergence to obtain the network model with the optimal parameters as the trained remote sensing image semantic segmentation model;

[0050] The test set is used to verify the performance of the trained remote sensing image semantic segmentation model.

[0051] The present invention provides a remote sensing image semantic segmentation method based on a hyperbolic neural network. Compared with the existing technology, the present invention has the following advantages and beneficial effects:

[0052] The present invention constructs a hyperbolic space neural network model based on an encoder-decoder structure. The neural network maps convolutional layers, batch normalization, nonlinear activation, residual connections, Softmax functions and deconvolution layers to hyperbolic space by adopting a Lorentz model to learn low-distortion remote sensing image representations. By mapping features to hyperbolic space and Euclidean space and utilizing the characteristics of hyperbolic space, the hierarchical structure and spatial relationship can be more naturally represented, and contextual dependencies are effectively integrated. The hierarchical structure and spatial relationship in the remote sensing image can be more effectively captured, thereby achieving more accurate semantic segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A remote sensing image semantic segmentation model architecture based on a hyperbolic neural network provided by an embodiment of the present invention;

[0054] Figure 2 This is a hyperbolic coding block data processing process provided by an embodiment of the present invention;

[0055] Figure 3 It is the data processing process of the Euclidean similarity enhanced hyperbolic attention module provided by an embodiment of the present invention;

[0056] Figure 4 It is a schematic diagram of the LoveDA data set experimental data and results in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0058] It should be emphasized here that the step marks mentioned below are not intended to limit the order of the steps, but it should be understood that the steps can be executed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be executed simultaneously.

[0059] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0060] The specific embodiments of the present invention are described in further detail below in conjunction with the accompanying drawings. The following examples are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0061] The present invention provides a remote sensing image semantic segmentation method based on a hyperbolic neural network, comprising:

[0062] Acquire a target remote sensing image, which includes remote sensing images of target objects of interest (such as buildings, forests, water bodies, etc.) and other objects as background (such as roads, farmlands, etc.);

[0063] The acquired target remote sensing image data is input into a pre-trained remote sensing image semantic segmentation model based on a hyperbolic neural network, and finally a prediction map containing semantic information, namely the semantic segmentation result, is output; in this prediction map, different target objects or areas are divided into different semantic categories and marked with corresponding colors or labels.

[0064] It should be noted that target features refer to specific types of features that are the focus of the study, such as buildings, forests, and water bodies;

[0065] Other features refer to other features that exist in remote sensing images but are not the focus of research, such as roads, farmland, and urban infrastructure. Although these features appear in the image, they are not the main objects of analysis.

[0066] In the present invention, a remote sensing image semantic segmentation model based on a hyperbolic neural network is pre-trained, specifically as follows:

[0067] S1. Obtain an earth observation remote sensing image dataset, which contains remote sensing images of target objects of interest (such as buildings, forests, water bodies, etc.) and other objects as background, and randomly divide the training set and the test set according to a fixed ratio;

[0068] S2. constructing a hyperbolic spatial neural network model based on an encoder-decoder structure, wherein the network model includes a hyperbolic encoder consisting of five hyperbolic encoding blocks and a hyperbolic decoder consisting of five hyperbolic decoding blocks;

[0069] S3, inputting the training set images into the constructed hyperbolic space neural network model for training until the model converges, and obtaining a network model with optimal parameters as a trained remote sensing image semantic segmentation model;

[0070] S4. Use the test set to verify the performance of the constructed remote sensing image semantic segmentation model based on hyperbolic neural network.

[0071] Specifically, the architecture of the hyperbolic space neural network model based on the encoder-decoder structure is as follows: Figure 1 As shown, the training process of the hyperbolic space neural network model is as follows:

[0072] S31. Assume that the input original remote sensing image is .in, Represent the height, width and number of channels of the image respectively; the original remote sensing image is mapped to the hyperbolic space through the logarithmic mapping function to obtain the input hyperbolic feature ;

[0073] S32, the hyperbolic feature Input into the hyperbolic encoder consisting of five hyperbolic encoding blocks to obtain the hyperbolic encoding features ,in , , They represent the hyperbolic encoder The height, width, and number of channels of the feature map output by the hyperbolic encoding block;

[0074] S33: output the hyperbolic coding feature of the fifth hyperbolic coding block obtained in S32 above Input into the hyperbolic decoder composed of five hyperbolic decoding blocks to obtain the hyperbolic decoding features ,in , , They represent the hyperbolic encoder The height, width and number of channels of the feature map output by the hyperbolic decoding block are consistent with the size of the feature map output by the corresponding hyperbolic encoding block;

[0075] S34, using Lorentz convolution (LConv) coupled with Lorentz normalized exponential function (L-Softmax) as the hyperbolic decoding feature output by the first hyperbolic decoding block above Assign a semantic label to each pixel as the output result of semantic segmentation.

[0076] In step S32, the hyperbolic coding block data processing process is as follows:

[0077] The feature map is obtained by convolution, Lorentz batch normalization and nonlinear activation of the input features in hyperbolic space. ;

[0078] The feature map The input is a hyperbolic attention module enhanced by Euclidean similarity, and residual connections are used to enhance the focus on important features;

[0079] Finally, average pooling is applied to downsample the features enhanced by residual connection to reduce the spatial resolution of the feature map and obtain the output of the hyperbolic encoding block.

[0080] It should be noted that each hyperbolic encoding block receives the hyperbolic encoding features output by the hyperbolic encoding block of the previous layer as input. In particular, the input features of the first hyperbolic encoding block are the hyperbolic features initialized by the hyperbolic features. The output of the last hyperbolic encoding block is taken as the final output of the hyperbolic encoder and is input into the hyperbolic decoder.

[0081] The data processing process of the hyperbolic decoding block is the same as that of the hyperbolic encoding block.

[0082] It should be noted that the first hyperbolic decoding block of the hyperbolic decoder receives the output features of the last hyperbolic encoding block of the hyperbolic encoder. Each subsequent decoding block receives the output of the previous decoding block; each hyperbolic decoding block outputs the processed hyperbolic decoding features as the input of the next hyperbolic decoding block. In particular, the output of the last hyperbolic decoding block will be used for the final semantic label assignment.

[0083] In one embodiment, the structure of the hyperbolic encoding block is as follows: Figure 2 As shown, the data processing process is as follows:

[0084] S321. Perform convolution operations on input features in hyperbolic space and extract local features using Lorentz convolution layers , expressed as: ,in and Represent the Lorentz full convolution layer and cascade operation respectively, and the formula is as follows:

[0085] ,

[0086] .

[0087] in:

[0088] is a function used to modify the affine transformation. It introduces curvature Constraints are used to ensure the validity of affine transformations in hyperbolic space;

[0089] Represents the weight matrix in affine transformation;

[0090] Represents the bias vector in the affine transformation, which is used to adjust the linear combination of the input data;

[0091] Represents the curvature of the hyperbolic space, which is a parameter that controls the degree of curvature of the geometric space;

[0092] Indicates the number of input data that needs to be processed;

[0093] Indicates The square of the input points is used to calculate the modulus of the input data;

[0094] and represents the transpose of the input vector;

[0095] and Indicates the height and width of the current position of the input feature map;

[0096] Indicates the offset position of the height and width of a certain position in the image;

[0097] Indicates the offset step of the convolution window, that is, the movement in the height and width directions based on the current window;

[0098] Represents the step size of the convolution, that is, the distance the convolution kernel slides on the input image;

[0099] Represents the height and width of the convolution window, that is, the size of the convolution kernel.

[0100] S322, extracting local features Perform Lorentz batch normalization to obtain features , expressed as: , of which indivual The features obtained from the output feature map of Composition of batches The center point The formula is as follows:

[0101] ,

[0102] in: are the weights used for batch normalization; K represents the curvature of the hyperbolic space.

[0103] In the above formula, a set of features is generated by convolution operation on the input features in hyperbolic space. These features can be regarded as The result is processed in hyperbolic space, and each This corresponds to a point or pixel in the feature map. The features in a batch usually represent multiple local regions or samples of the input image or other input data.

[0104] S323, normalized features Use Lorentz nonlinear activation function to perform nonlinear activation to obtain features , expressed as: ,in Represents the spatial components of the input vector.

[0105] S324, the characteristics Input to the Euclidean similarity enhanced hyperbolic attention module to obtain the feature , the Euclidean similarity enhanced hyperbolic attention module is as follows Figure 3 As shown, the data processing process is as follows:

[0106] The input features Input to both the hyperbolic branch and the Euclidean branch;

[0107] In hyperbolic space, for the input features After performing Lorentz convolution, batch normalization, and nonlinear activation operations in sequence, the inverse Lorentz distance between features is calculated, and the inverse Lorentz distance is normalized using Lorentz Softmax (L-Softmax) to generate a hyperbolic attention map , expressed as:

[0108] ;

[0109] in, Represents the Lorentz distance, which is used to calculate the input point and reference point The distance between Indicates the total number of reference points when calculating the Lorentz distance.

[0110] The input features Use logarithmic mapping to map from hyperbolic space to Euclidean space, use standard convolution layer (Conv) to perform convolution operation on the mapped features, and use standard Softmax function to normalize the Euclidean feature map to generate Euclidean attention map , expressed as:

[0111] ;

[0112] in: Represents the Euclidean distance, which is used to calculate the input point and reference point The distance between.

[0113] The obtained hyperbolic attention map and Euclidean attention map Through weighted summation, we get the Euclidean similarity enhanced hyperbolic attention map , expressed as:

[0114] .

[0115] Finally, the input features Euclidean similarity enhanced hyperbolic attention graph After feature aggregation Agg, residual connection is used to obtain the output features of the Euclidean similarity enhanced hyperbolic attention module , where the feature aggregation Agg is expressed as:

[0116] ;

[0117] in: Represents the attention weight, which is used to determine the feature For features The importance of represents the Lorentz norm, which is used to measure distance in hyperbolic space.

[0118] S325, features obtained from S323 And the features enhanced by S324 Euclidean similarity enhanced hyperbolic attention module Obtaining features using Lorentz residual connections , in order to retain key feature information and avoid the gradient disappearance problem in deep networks, expressed as: ,in Represents the transformations applied at each layer of the network, typically involving nonlinear activations, batch normalization, and fully convolutional layers; LADD is the Lorentz summation function, with the following formula:

[0119] ;

[0120] S326, repeat the steps from S321 to S325, that is, the features obtained in S325 After Lorentz convolution, normalization, activation, Euclidean similarity enhancement and residual connection operations, the features are obtained again. ;

[0121] S327, apply Lorentz average pooling (LAP) to the features Downsampling is performed to reduce the spatial resolution of the feature map. Unlike the Euclidean average pooling that directly calculates the arithmetic mean of the features, the Lorentz average pooling calculates the hyperbolic mean, which is usually called the Karcher mean in hyperbolic geometry and is expressed as: ,

[0122] After Lorentz average pooling, the feature output is .

[0123] It should be noted that the input feature of the first hyperbolic encoding block is ; Input to The characteristics of the hyperbolic coding blocks are The output features of the hyperbolic encoding block ,in .

[0124] The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder composed of five hyperbolic decoding blocks to obtain the hyperbolic decoding features ,in , , Respectively represent the hyperbolic decoder The height, width and number of channels of the feature map output by the hyperbolic decoding block. The operation of the hyperbolic decoding block is similar to that of the hyperbolic encoding block, so it will not be repeated here.

[0125] It should be noted that the input feature of the fifth hyperbolic decoding block is , enter into The characteristics of the hyperbolic decoding blocks are The output features of the hyperbolic decoding block ,in .

[0126] In one embodiment, the training process of the hyperbolic space neural network model based on the encoder-decoder structure is as follows:

[0127] A31. Divide the LoveDA dataset into training and test sets, input the training set images into the neural network, set the number of training rounds to 500, the batch size to 64, and use 64 images each time to update the model parameters;

[0128] A32. At the beginning of training, the model uses an initial learning rate of 0.02. The learning rate will be gradually reduced through a step-by-step decay strategy, combined with a momentum of 0.9 to accelerate training convergence;

[0129] A33, using Riemannian SGD optimizer to accommodate the hyperbolic geometry in the embedding layer, with curvature set to -1. Two Nvidia A40 48GB GPUs for parallel processing to accelerate training;

[0130] A34. The entire training process uses the hyperbolic normalized cross entropy loss function to optimize the model's output so that it gradually approaches the true label. As the training progresses, the network parameters are continuously adjusted in 500 rounds until the loss value gradually decreases and the model finally reaches a convergence state, that is, the training is completed, the optimal parameter configuration is obtained, and a trained remote sensing image semantic segmentation model based on a hyperbolic neural network is obtained.

[0131] The verification process of the remote sensing image semantic segmentation model is as follows:

[0132] A41. The test set in the LoveDA dataset is used to evaluate the constructed remote sensing image semantic segmentation model based on hyperbolic neural network. The dataset contains 835 1024x1024 pixel images, covering urban and rural areas of Nanjing, Changzhou and Wuhan;

[0133] A42. By comparing the model prediction results with the true labels, the segmentation performance of the model in different categories is evaluated, and then the generalization ability of the model on large-scale remote sensing images is verified. The verification results are as follows: Figure 4 As shown. It can be seen that the hyperbolic space neural network model based on the encoder-decoder structure performs well on the LoveDA dataset. The model can clearly divide buildings and roads, accurately distinguish agricultural land from surrounding landscapes, and ensure consistent segmentation effects.

[0134] Based on the above inventive concept, the present invention also provides a remote sensing image semantic segmentation device based on a hyperbolic neural network, which is used to implement the above remote sensing image semantic segmentation method based on a hyperbolic neural network. The device includes:

[0135] A data acquisition module is used to acquire a target remote sensing image, wherein the target remote sensing image includes a target object of interest and a background object;

[0136] A semantic output module is used to input the acquired target remote sensing image data into a pre-trained remote sensing image semantic segmentation model based on a hyperbolic neural network, and output a prediction map containing semantic information, i.e., a semantic segmentation result; in the prediction map, different target objects or regions are divided and labeled into different semantic categories;

[0137] The remote sensing image semantic segmentation model based on the hyperbolic neural network includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; the hyperbolic encoding block and the hyperbolic decoding block process and fuse the input features in the hyperbolic space and the Euclidean space, and enhance the hierarchical structure and spatial relationship of the feature map.

[0138] Preferably, the device also includes a model training module,

[0139] The model training module is specifically used to:

[0140] Obtaining an earth observation remote sensing image data set, and randomly dividing it into a training set and a test set according to a ratio; the earth observation remote sensing image contains a target object of interest and a background object;

[0141] Constructing a hyperbolic space neural network model based on an encoder-decoder structure, wherein the hyperbolic space neural network model includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks;

[0142] The training set image is input into the hyperbolic space neural network model. In the hyperbolic space neural network model, the input original earth observation remote sensing image is mapped to the hyperbolic space through a logarithmic mapping function to obtain the input hyperbolic feature ; The hyperbolic feature Input into the hyperbolic encoder, and each hyperbolic encoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic encoding features , ; The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder, and each hyperbolic decoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic decoding features , ; Use Lorentz convolution coupled with Lorentz normalized exponential function as the hyperbolic decoding feature output of the first hyperbolic decoding block Assign a semantic label to each pixel; compare the model output with the true label, and use the hyperbolic normalized cross entropy loss function to optimize the model parameters so that the model output gradually approaches the true label, and finally reach model convergence to obtain the network model with the optimal parameters as the trained remote sensing image semantic segmentation model;

[0143] The test set is used to verify the performance of the trained remote sensing image semantic segmentation model.

[0144] It is worth pointing out that the device embodiment corresponds to the above-mentioned method embodiment, and the implementation methods of the above-mentioned method embodiments are all applicable to the device embodiment and can achieve the same or similar technical effects, so they will not be repeated here.

[0145] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0146] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A remote sensing image semantic segmentation method based on hyperbolic neural network, characterized in that: include: Acquire a target remote sensing image, wherein the target remote sensing image includes a target object of interest and background objects; Inputting the acquired target remote sensing image data into a pre-trained remote sensing image semantic segmentation model based on a hyperbolic neural network, and outputting a prediction map containing semantic information, i.e., a semantic segmentation result; in the prediction map, different target objects or regions are divided and labeled into different semantic categories; The remote sensing image semantic segmentation model based on the hyperbolic neural network includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; the hyperbolic encoding block and the hyperbolic decoding block process and fuse the input features in the hyperbolic space and the Euclidean space, and enhance the hierarchical structure and spatial relationship of the feature map.

2. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 1, characterized in that: The training process of the remote sensing image semantic segmentation model based on the hyperbolic neural network is as follows: Obtaining an earth observation remote sensing image data set, and randomly dividing it into a training set and a test set according to a ratio; the earth observation remote sensing image contains a target object of interest and a background object; Constructing a hyperbolic space neural network model based on an encoder-decoder structure, wherein the hyperbolic space neural network model includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; The training set images are input into the hyperbolic space neural network model for training. During the training process, the model parameters are optimized using the hyperbolic normalized cross entropy loss function so that the model output gradually approaches the true label, and finally the model converges to obtain the network model with the optimal parameters as the trained remote sensing image semantic segmentation model; The test set is used to verify the performance of the trained remote sensing image semantic segmentation model.

3. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 2, characterized in that: The hyperbolic encoder comprises five hyperbolic encoding blocks, the five hyperbolic encoding blocks have the same structure and the same data processing method; The hyperbolic decoder includes five hyperbolic decoding blocks, and the five hyperbolic decoding blocks have the same structure and the same data processing method.

4. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 3, characterized in that: Inputting the training set images into the hyperbolic space neural network model for training, including: The input original earth observation remote sensing image is mapped to the hyperbolic space through the logarithmic mapping function to obtain the input hyperbolic feature ; The hyperbolic feature Input into the hyperbolic encoder, and each hyperbolic encoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic encoding features , ; The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder, and each hyperbolic decoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic decoding features , ; Hyperbolic decoding features output by the first hyperbolic decoding block using Lorentz convolution coupled with a Lorentz normalized exponential function Assign a semantic label to each pixel as the output result of semantic segmentation.

5. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 4, characterized in that: The hyperbolic feature The data is input into the hyperbolic encoder, and the data processing process of each hyperbolic encoding block in the hyperbolic encoder is as follows: The hyperbolic features of the input in the hyperbolic space Perform convolution operation and use Lorentz convolution layer to extract local features ; For the local features Perform Lorentz batch normalization to obtain features ; After normalization, the features Use Lorentz nonlinear activation function to perform nonlinear activation to obtain features ; The characteristics Input into the Euclidean similarity enhanced hyperbolic attention module to obtain the feature ; The characteristics and the characteristics Obtaining features using Lorentz residual connections ; The characteristics The above method is used again to perform Lorentz convolution, Lorentz batch normalization, nonlinear activation, Euclidean similarity enhanced hyperbolic attention module processing and Lorentz residual connection to obtain the features ; Lorentz average pooling is used to Downsample to get the output of the hyperbolic encoding block , .

6. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 5, characterized in that: The Euclidean similarity enhanced hyperbolic attention module, the data processing process is as follows: The characteristics Input to both the hyperbolic branch and the Euclidean branch; In hyperbolic space, for the feature After performing Lorentz convolution, batch normalization, and nonlinear activation operations in sequence, the inverse Lorentz distance between features is calculated, and the inverse Lorentz distance is normalized using Lorentz Softmax to generate a hyperbolic attention map. ; The characteristics Use logarithmic mapping to map from hyperbolic space to Euclidean space, use standard convolutional layer to convolve the mapped features, and use standard Softmax function to normalize the convolved feature map to generate Euclidean attention map ; The hyperbolic attention map and the Euclidean attention map Through weighted summation, we get the Euclidean similarity enhanced hyperbolic attention map ; The characteristics and the Euclidean similarity enhanced hyperbolic attention graph After feature aggregation, residual connection is used to obtain the output features of the Euclidean similarity enhanced hyperbolic attention module .

7. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 6, characterized in that: In the hyperbolic encoder, the input of the first hyperbolic encoding block is the hyperbolic feature , No. The input of the hyperbolic encoding block is The output of the hyperbolic encoding block, .

8. The method for semantic segmentation of remote sensing images based on a hyperbolic neural network according to claim 7, characterized in that: The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder, the data processing process of each hyperbolic decoding block in the hyperbolic decoder is the same as the data processing process of the hyperbolic encoding block; In the hyperbolic decoder, the input of the fifth hyperbolic decoding block is the hyperbolic encoding feature , No. The input of the hyperbolic decoding block is The output of the hyperbolic decoding block, .

9. A remote sensing image semantic segmentation device based on a hyperbolic neural network, characterized in that: The device for implementing the remote sensing image semantic segmentation method based on a hyperbolic neural network according to any one of claims 1 to 8 comprises: A data acquisition module is used to acquire a target remote sensing image, wherein the target remote sensing image includes a target object of interest and background objects; A semantic output module is used to input the acquired target remote sensing image data into a pre-trained remote sensing image semantic segmentation model based on a hyperbolic neural network, and output a prediction map containing semantic information, i.e., a semantic segmentation result; in the prediction map, different target objects or regions are divided and labeled into different semantic categories; The remote sensing image semantic segmentation model based on the hyperbolic neural network includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; the hyperbolic encoding block and the hyperbolic decoding block process and fuse the input features in the hyperbolic space and the Euclidean space, and enhance the hierarchical structure and spatial relationship of the feature map.

10. The remote sensing image semantic segmentation device based on hyperbolic neural network according to claim 9, characterized in that: The device also includes a model training module, The model training module is specifically used to: Obtaining an earth observation remote sensing image data set, and randomly dividing it into a training set and a test set according to a ratio; the earth observation remote sensing image contains a target object of interest and a background object; Constructing a hyperbolic space neural network model based on an encoder-decoder structure, wherein the hyperbolic space neural network model includes a hyperbolic encoder composed of hyperbolic encoding blocks and a hyperbolic decoder composed of hyperbolic decoding blocks; The training set image is input into the hyperbolic space neural network model. In the hyperbolic space neural network model, the input original earth observation remote sensing image is mapped to the hyperbolic space through a logarithmic mapping function to obtain the input hyperbolic feature ; The hyperbolic feature Input into the hyperbolic encoder, and each hyperbolic encoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic encoding features , ; The hyperbolic encoding features output by the fifth hyperbolic encoding block Input into the hyperbolic decoder, and each hyperbolic decoding block processes the input features in the hyperbolic space and the Euclidean space to obtain the hyperbolic decoding features , ; Use Lorentz convolution coupled with Lorentz normalized exponential function as the hyperbolic decoding feature output of the first hyperbolic decoding block Assign a semantic label to each pixel; compare the model output with the true label, and use the hyperbolic normalized cross entropy loss function to optimize the model parameters so that the model output gradually approaches the true label, and finally reach model convergence to obtain the network model with the optimal parameters as the trained remote sensing image semantic segmentation model; The test set is used to verify the performance of the trained remote sensing image semantic segmentation model.

Images