Instance segmentation method and device of remote sensing image and electronic equipment

Abstract

The invention relates to an instance segmentation method and device for a remote sensing image and electronic equipment. The method comprises the following steps: acquiring a to-be-processed remote sensing image; the remote sensing image is input into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, and the feature extraction backbone network comprises a block embedding layer and a plurality of Mama modules; and inputting the remote sensing image feature map into an image instance segmentation network of the image instance segmentation model to obtain an instance segmentation result of the remote sensing image output by the image instance segmentation network, thereby constructing a feature extraction backbone network through a block embedding layer and a plurality of Mama modules, and obtaining an instance segmentation result of the remote sensing image. The feature extraction backbone network can have global situation modeling capability and linear time complexity at the same time, and the accuracy and reliability of obtaining the remote sensing image feature map are improved, so that the accuracy and efficiency of obtaining the instance segmentation result of the remote sensing image are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a method, device and electronic device for instance segmentation of remote sensing images. Background Art

[0002] Remote sensing image instance segmentation is of great significance. The instance segmentation results of remote sensing images have wide applications and far-reaching impacts in many fields such as earth observation, environmental monitoring, and urban planning. Among the related technologies, instance segmentation of remote sensing images is mainly based on Convolutional Neural Networks (CNN) and Transformer. However, CNN usually uses local receptive fields and often finds it difficult to capture the global context of remote sensing images. The computational complexity of Transformer increases quadratically with the increase of input image size, resulting in huge consumption of computing resources and low reasoning efficiency. Therefore, how to improve the accuracy and efficiency of instance segmentation of remote sensing images has become an urgent problem to be solved. Summary of the invention

[0003] The present disclosure provides a method, device and electronic device for instance segmentation of remote sensing images, so as to at least solve the problem of low accuracy and efficiency of instance segmentation results of remote sensing images in the related art.

[0004] The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a method for instance segmentation of a remote sensing image is provided, the method comprising: obtaining a remote sensing image to be processed; inputting the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network; inputting the remote sensing image feature map into an image instance segmentation network of the image instance segmentation model to obtain an instance segmentation result of the remote sensing image output by the image instance segmentation network.

[0006] According to a second aspect of an embodiment of the present disclosure, a remote sensing image instance segmentation device is provided, comprising: an acquisition module for acquiring a remote sensing image to be processed; a first processing module for inputting the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network comprises a block embedding layer and multiple Mamba modules; a second processing module for inputting the remote sensing image feature map into an image instance segmentation network of the image instance segmentation model to obtain an instance segmentation result of the remote sensing image output by the image instance segmentation network.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the instructions to implement the instance segmentation method of a remote sensing image as described in the first aspect of an embodiment of the present disclosure.

[0008] According to the fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the instance segmentation method of remote sensing images as described in the first aspect of the embodiment of the present disclosure.

[0009] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0010] The embodiment of the present disclosure obtains a remote sensing image to be processed, inputs the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, and inputs the remote sensing image feature map into an image instance segmentation network of an image instance segmentation model to obtain an instance segmentation result of the remote sensing image output by the image instance segmentation network. Therefore, the present disclosure constructs a feature extraction backbone network through a block embedding layer and multiple Mamba modules. The feature extraction backbone network can simultaneously have global situation modeling capabilities and linear time complexity, thereby improving the accuracy and reliability of obtaining remote sensing image feature maps, thereby improving the accuracy and efficiency of obtaining instance segmentation results of remote sensing images.

[0011] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0013] Figure 1 The figure is a flowchart of a method for instance segmentation of a remote sensing image according to an exemplary embodiment.

[0014] Figure 2 The present invention is a flowchart of a method for instance segmentation of a remote sensing image according to another exemplary embodiment.

[0015] Figure 3 It is a schematic diagram of a process of obtaining a remote sensing image feature map according to another exemplary embodiment.

[0016] Figure 4 The present invention is a flowchart of a method for instance segmentation of a remote sensing image according to another exemplary embodiment.

[0017] Figure 5 is a flow chart showing an SSM module adaptive scanning according to another exemplary embodiment.

[0018] Figure 6 The present invention is a flowchart of a method for instance segmentation of a remote sensing image according to another exemplary embodiment.

[0019] Figure 7 The present invention is a flowchart of an image instance segmentation model training according to another exemplary embodiment.

[0020] Figure 8 is a schematic diagram showing a rotation angle according to another exemplary embodiment.

[0021] Fig. 9 The figure is a flowchart of a method for instance segmentation of a remote sensing image according to another exemplary embodiment.

[0022] Fig.10 The invention is a block diagram of a device for segmenting an instance of a remote sensing image according to another exemplary embodiment.

[0023] Fig.11 is a block diagram of an electronic device according to another exemplary embodiment. DETAILED DESCRIPTION

[0024] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0026] The example segmentation method, device and electronic device for remote sensing images proposed in the present disclosure are described in detail using embodiments.

[0027] Figure 1 A schematic flow chart of a method for instance segmentation of a remote sensing image provided in an embodiment of the present disclosure.

[0028] like Figure 1As shown, the instance segmentation method of remote sensing images proposed in this embodiment includes the following steps:

[0029] S101, obtaining a remote sensing image to be processed.

[0030] The remote sensing image to be processed is any remote sensing image that requires instance segmentation.

[0031] It should be noted that the present disclosure does not limit the specific method for obtaining the remote sensing image to be processed, and it can be selected according to actual conditions.

[0032] Optionally, remote sensing images may be collected by a remote sensing image collection device, and the collected remote sensing images may be used as remote sensing images to be processed.

[0033] Optionally, remote sensing images may be selected from a remote sensing image database, and the selected remote sensing images may be used as remote sensing images to be processed.

[0034] S102, inputting the remote sensing image into the feature extraction backbone network of the image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and multiple Mamba modules.

[0035] Among them, the image instance segmentation model is pre-trained.

[0036] In the disclosed embodiment, the feature extraction backbone network of the image instance segmentation model includes a block embedding layer and multiple Mamba modules.

[0037] It should be noted that the present disclosure does not limit the number of Mamba modules, which can be set according to actual conditions.

[0038] Optionally, the number of Mamba modules may be set to 5; optionally, the number of Mamba modules may be set to 4.

[0039] In the disclosed embodiment, after acquiring the remote sensing image, the remote sensing image can be input into the block embedding layer to obtain the embedding vector sequence output by the block embedding layer, and the embedded vector sequence is input into the first Mamba module among multiple Mamba modules, and the first Mamba module performs feature extraction on the embedded vector sequence to obtain a first remote sensing image feature map, and the first remote sensing image feature map is input into the second Mamba module, and the second Mamba module continues to perform feature extraction on the first remote sensing image feature map to obtain a second remote sensing image feature map, and the above process is repeated until the remote sensing image feature map is output by the last Mamba module.

[0040] It should be noted that through the feature extraction backbone network, the selective scanning mechanism of the Mamba module can be used to effectively obtain the global context in the remote sensing image. The feature extraction backbone network has linear time complexity in the training and inference process, which can effectively improve the accuracy and efficiency of remote sensing image instance segmentation.

[0041] S103, inputting the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network.

[0042] It should be noted that the image instance segmentation network can be determined by Mask Regional Convolutional Neural Networks (Mask-RCNN for short).

[0043] In the disclosed embodiment, the remote sensing image feature map can be input into the Region Proposal Network (RPN) to obtain the candidate bounding box corresponding to the remote sensing image feature map output by the RPN, and feature extraction is performed on the candidate bounding box to obtain a target feature map of the same size. The target feature map is input into the classification regression branch and the mask branch respectively, and the classification regression branch outputs the category information and position information of the candidate bounding box, and the mask branch outputs the segmentation mask of the candidate bounding box. According to the category information, position information and segmentation mask, the instance segmentation result of the remote sensing image is determined.

[0044] In summary, the instance segmentation method of the remote sensing image provided by the embodiment of the present disclosure obtains the remote sensing image to be processed, inputs the remote sensing image into the feature extraction backbone network of the image instance segmentation model, so as to obtain the remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and multiple Mamba modules, and inputs the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model, so as to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network. Therefore, the present disclosure constructs a feature extraction backbone network through the block embedding layer and multiple Mamba modules, and the feature extraction backbone network can simultaneously have the global situation modeling capability and linear time complexity, thereby improving the accuracy and reliability of obtaining the remote sensing image feature map, thereby improving the accuracy and efficiency of obtaining the instance segmentation result of the remote sensing image.

[0045] As a possible implementation, Figure 2 As shown, based on the above embodiment, the remote sensing image is input into the feature extraction backbone network of the image instance segmentation model to obtain the specific process of the remote sensing image feature map output by the feature extraction backbone network, including the following steps:

[0046] S201, inputting the remote sensing image into the block embedding layer to obtain an embedding vector sequence output by the block embedding layer.

[0047] For example, the remote sensing image x to be processed is input into the patch embedding layer, and the patch embedding layer transforms the remote sensing image into an embedding vector sequence consisting of a fixed-length vector Among them, d is the dimension of the embedding vector, and N is the number of patches. Specifically, the Patch embedding layer divides the remote sensing image into image patches according to the patch size p×p, and converts each image patch into a vector of the same length through a convolutional network, thereby realizing the serialization of remote sensing image features and laying the foundation for the subsequent scanning of the Mamba module.

[0048] S202, input the embedded vector sequence to the first Mamba module among the multiple Mamba modules, the first Mamba module performs feature extraction on the embedded vector sequence to obtain a first remote sensing image feature map, and the first remote sensing image feature map is input to the second Mamba module, the second Mamba module continues to perform feature extraction on the first remote sensing image feature map to obtain a second remote sensing image feature map, and the above process is repeated until the last Mamba module outputs the remote sensing image feature map.

[0049] In the disclosed embodiment, each Mamba module includes a first linear layer, a second linear layer, a deep convolutional layer, a first activation layer, a second activation layer, a multi-state space submodule, a Gate module, a third linear layer and a downsampling layer.

[0050] For example, Figure 3 As shown, the remote sensing image can be input into the block embedding layer, and the embedding vector sequence can be input into the Mamba module for processing to output the remote sensing image feature map, wherein the specific process of inputting the embedding vector sequence into the Mamba module for processing can be referred to the following embodiment.

[0051] As a possible implementation, Figure 4 As shown, based on the above embodiment, the first Mamba module performs feature extraction on the embedded vector sequence to obtain the specific process of the first remote sensing image feature map, including the following steps:

[0052] S401, inputting the embedding vector sequence into a first linear layer and a first activation layer connected sequentially to obtain a first feature vector output by the first activation layer.

[0053] It should be noted that after obtaining the embedding vector sequence, the embedding vector sequence is input into the first linear layer (projection layer), and each embedding vector is converted into a vector z of dimension αd by the expansion factor α, where z = Linear(x), x∈R d , z∈R αd , and then input the vector z into the first activation layer to obtain the first eigenvector output by the first activation layer.

[0054] S402, inputting the embedding vector sequence into a second linear layer, a deep convolutional layer, and a second activation layer connected in sequence to obtain a second feature vector output by the activation layer.

[0055] It should be noted that after obtaining the embedding vector sequence, the embedding vector sequence is input into the second linear layer (projection layer), and each embedding vector is converted into a vector u of dimension αd by the expansion factor α, where u = Linear(x), x∈R d , u∈R αd , then input the vector u into the depth convolution layer, and input the output of the depth-wise (DW) convolution layer into the second activation layer to obtain the second feature vector v output by the second activation layer.

[0056] For example, the second feature vector v=act[DW(Conv(u))], where DWConv(·) is a deep convolution operation and act(·) is an activation operation. The dimension of the vector does not change after the deep convolution operation and the activation operation.

[0057] S403, input the second feature vector to the multi-state space submodule to obtain the third feature vector output by each state space submodule, and input the third feature vector to the Gate module for fusion to obtain the fused feature vector output by the Gate module.

[0058] It should be noted that the Multi-State Space Model (Multi-SSM) expands the Selective State Space Module (S6) to 6 branches for scanning. The state space module is a tool for modeling and analyzing dynamic systems. The SSM can describe the evolution of the system and estimate the internal state of the system through observation data. Convert to a one-dimensional output signal The SSM consists of a state equation and an observation equation:

[0059] h(t)=Ah(t-1)+Bv(t)

[0060] f(t)=Ch(t)+Dv(t)

[0061] in, is the hidden state at time t, A is the state transfer matrix, which describes the transfer amount of the state from time t-1 to time t, B is the input control vector, which determines the influence of the input v(t) on the hidden state, and C is the observation matrix, which describes the mapping of the state to the output. Usually D provides a direct signal from input to output (jump connection), which can be simplified by omitting D.

[0062] It should be noted that SSM uses the sampling time scale parameter Δ to convert the continuous SSM into a discrete SSM (S4). Therefore, the mapping from the continuous signal input v(t) to the output f(t) is converted to a sequence-to-sequence mapping, and the discretization uses the sampling time scale parameter Δ to convert the continuous parameters (A, B) into discrete parameters Zero-Order Hold (ZOH) is usually used to complete the conversion.

[0063] It should be noted that S4 is a linear time-invariant state space model. Due to the lack of content perception, it cannot achieve selective tasks based on input content. However, in fact, it is more effective to invest more energy in important content and dynamically adjust the importance level to match the complex input content. The selective state space module (S6), namely the Mamba model, is proposed for this purpose. Specifically, in order to operate on an input sequence with a batch size of b, a length of l, and a number of channels of d, Mamba applies SSM independently to each channel. In Mamba, the matrices B, C, and Δ in S4 are functions of the input, allowing the model to be adaptively adjusted according to the input. The discretization process after incorporating the selection mechanism into the model is as follows:

[0064]

[0065]

[0066] Δ=τ A (Parameter+S A (v))

[0067] Where, N is the number of states, S B (v) and S c (v) is a linear function that projects the input into an N-dimensional vector, which is transformed into a time-varying system that depends on the input through the above operation S6.

[0068] It should be noted that the S6 model was originally designed for natural language processing problems, and the correlation between different pixels in an image is one-dimensional and cannot be directly applied to the field of image processing. By extending the scanning strategy in S6 to multiple directions such as horizontal, vertical, flipped horizontal, and flipped vertical, the Mamba model was applied to the field of image processing. Although the computational efficiency has been improved, it still cannot achieve the expected accuracy.

[0069] It should be noted that in order to improve the accuracy of segmentation results of remote sensing images, the present disclosure proposes an adaptive rotation variable scanning strategy, and improves the expression ability of the feature extraction backbone network by constructing a Multi-SSM module.

[0070] In an embodiment of the present disclosure, a target scanning strategy associated with each state space submodule can be obtained, wherein the target scanning strategy includes a horizontal scanning strategy, a flipped horizontal scanning strategy, a vertical scanning strategy, a flipped vertical scanning strategy, a rotationally variable scanning strategy, and a flipped rotationally variable scanning strategy. According to the target scanning strategy, the third eigenvector output by each state space submodule is obtained.

[0071] In the embodiment of the present disclosure, the number of state space submodules may be 6, and each state space submodule is associated with a target scanning strategy.

[0072] It should be noted that, for the rotation variable scanning strategy and the flip rotation variable scanning strategy, a plurality of non-overlapping windows of fixed size pre-divided by the remote sensing image can be obtained, and a second eigenvector matching each non-overlapping window can be obtained, and a routing function is constructed according to the global average pooling operation and the linear transformation operation, and the second eigenvector is input into the routing function. Through the routing function, the transformation parameters of each non-overlapping window are predicted, wherein the transformation parameters include scale, displacement and angle. According to the transformation parameters, each non-overlapping window is transformed into a rotation variable window, and within the rotation variable window, the rotation variable scanning strategy and the flip rotation variable scanning strategy are executed to obtain the fourth eigenvector output by each state space submodule.

[0073] For example, if the size of each non-overlapping window is ws, for any non-overlapping window w i , the routing function is defined as a concatenation of a global pooling operation and a linear transformation operation:

[0074] S,O,θ=Linear(LeakyReLu(GAP(w)))

[0075] Among them, S, O, and θ are the transformations of the non-overlapping window w in scale, displacement, and angle, respectively.

[0076] Among them, the transformed non-overlapping window position is:

[0077]

[0078] Among them, (x c ,y c ) is the center position of the non-overlapping window, (o x ,o y ) is the amount of change in the center position, (x l ,y l ) and (x r ,y r ) are the upper left corner and lower right corner positions of the non-overlapping window, is (x c ,y c ) the distance from the left border, top border, right border and bottom border, (s x ,s y ) is the scale variable of the non-overlapping window, (x′ l ,y′ l ) and (x′ r ,y′ r ) are the upper left and lower right corner positions of the transformed non-overlapping window.

[0079] For example, Figure 5 As shown in the figure, if the remote sensing image is pre-divided into 4 fixed-size non-overlapping windows (w1, w2, w3, w4), the transformation parameters (s i ,o i ,θ i ), transform each non-overlapping window into a rotationally variable window according to the transformation parameters, and obtain the features {w′ in all rotationally variable windows by bilinear interpolation. i}, an adaptive rotation variable scanning strategy is used to scan in two directions in all non-overlapping windows, namely, scanning in two directions of rotation variable scanning and flip rotation variable scanning. Through the above method, it is possible to adaptively select local areas and directions for scanning, thereby being able to better extract features from objects of any size and direction.

[0080] In the embodiment of the present disclosure, after the third feature vector is obtained, the third feature vector is input into the Gate module for fusion to obtain a fused feature vector output by the Gate module.

[0081] Optionally, a weight corresponding to each third feature vector may be obtained, and a weighted sum may be performed on each third image feature vector and the weight corresponding to each third image feature vector to obtain a fused feature vector.

[0082] For example, the Gate module can adaptively adjust the weights of different positions and channels, thereby further improving the feature expression capability of the feature extraction backbone network. For the second feature vector v input to the Multi-SSM module, the third feature vector obtained through the six target scanning strategies is The Gate module sets a learnable weight for each type of feature Therefore, the output of the Multi-SSM module can be:

[0083]

[0084] v′ i =SSM i (v)

[0085] Where, F is the fusion feature vector, i is the number of state space modules, SSM i (·) is the i-th state space module, each state space module corresponds to a target scanning strategy, v′ i is the third eigenvector.

[0086] S404, performing residual connection on the first feature vector and the fused feature vector, and inputting the target feature vector after the residual connection into the third linear layer and the downsampling layer to obtain a first remote sensing image feature map.

[0087] In the embodiment of the present disclosure, after obtaining the first eigenvector and the fused eigenvector, the first eigenvector and the fused eigenvector are residually connected, and the target eigenvector after the residual connection is input into the third linear layer for processing, and the output of the third linear layer is input into the downsampling layer for processing to obtain the first remote sensing image feature map.

[0088] It should be noted that the processing process of other Mamba modules can be found in the above embodiments and will not be described in detail here.

[0089] In summary, the instance segmentation method of the remote sensing image provided by the embodiment of the present disclosure inputs the embedding vector sequence into the first linear layer and the first activation layer connected in sequence to obtain the first feature vector output by the first activation layer, inputs the embedding vector sequence into the second linear layer, the deep convolution layer and the second activation layer connected in sequence to obtain the second feature vector output by the activation layer, inputs the second feature vector into the multi-state space submodule to obtain the third feature vector output by each state space submodule, and inputs the third feature vector into the Gate module for fusion to obtain the fused feature vector output by the Gate module, performs residual connection on the first feature vector and the fused feature vector, and inputs the target feature vector after the residual connection into the third The linear layer and the downsampling layer are used to obtain the first remote sensing image feature map. Therefore, the present invention constructs a feature extraction backbone network through the Mamba module, which can simultaneously have global situation modeling capabilities and linear time complexity, improves the accuracy and reliability of obtaining remote sensing image feature maps, and constructs an adaptive rotation variable scanning strategy in the feature extraction backbone network, which can adaptively learn the size, displacement and angle of the object according to the input, so as to adaptively learn the local area that needs to be scanned, improve the feature extraction backbone network's feature learning ability for objects with arbitrary directions and multi-scales, solve the problems of objects in remote sensing images with arbitrary directions and multi-scales, and ensure the accuracy and efficiency of the instance segmentation results of remote sensing images.

[0090] As a possible implementation, Figure 6 As shown, based on the above embodiment, the specific process of inputting the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network includes the following steps:

[0091] S601, inputting the remote sensing image feature map into the region proposal network RPN to obtain a candidate bounding box corresponding to the remote sensing image feature map output by the RPN.

[0092] It should be noted that Mask-RCNN usually uses a residual network (Residual Network, referred to as ResNet) as the feature extraction backbone network, but ResNet has a limited receptive field and it is difficult to capture global context information. Therefore, the present disclosure proposes to replace ResNet with a block embedding layer and multiple Mamba modules (Mamba model).

[0093] It should be noted that the region proposal network RPN is attached to the feature extraction backbone network to extract candidate regions (i.e., candidate bounding boxes) that may be instances from the remote sensing image feature map, where the candidate bounding boxes have different sizes and proportions to cover various instances appearing in the remote sensing image feature map.

[0094] S602, extract features from the candidate bounding box to obtain a target feature map of the same size.

[0095] In the disclosed embodiment, after the candidate bounding box is obtained, feature extraction is performed on the candidate bounding box, for example, a region of interest alignment (ROI Align) operation is performed on the candidate bounding box, and remote sensing image feature maps within candidate bounding boxes of different sizes can be converted into target feature maps of the same size.

[0096] S603: Input the target feature map to the classification and regression branch and the mask branch respectively, the classification and regression branch outputs the category information and position information of the candidate bounding box, and the mask branch outputs the segmentation mask of the candidate bounding box.

[0097] In the disclosed embodiment, after the target feature map is acquired, the target feature map is respectively input into the classification regression branch, and the classification regression branch outputs the category information and position information (coordinate information) of the candidate bounding box.

[0098] In the disclosed embodiment, after the target feature map is obtained, the target feature map is respectively input into the mask branch, and the mask branch outputs the segmentation mask of the candidate bounding box.

[0099] The segmentation mask is a binary image of the same size as the candidate bounding box, where the pixel value of the instance area is 1 and the pixel value of the background area is 0.

[0100] S604: Determine an instance segmentation result of the remote sensing image according to the category information, the location information and the segmentation mask.

[0101] In an embodiment of the present disclosure, for each candidate bounding box, after obtaining the category information, location information and segmentation mask, the category information, location information and segmentation mask can be combined to generate a segmentation result of the instance, and all instance segmentation results can be merged to determine the instance segmentation result of the remote sensing image.

[0102] In summary, the instance segmentation method of the remote sensing image provided by the embodiment of the present disclosure inputs the remote sensing image feature map into the region proposal network RPN to obtain the candidate bounding box corresponding to the remote sensing image feature map output by the RPN, performs feature extraction on the candidate bounding box, obtains the target feature map of the same size, inputs the target feature map into the classification regression branch and the mask branch respectively, the classification regression branch outputs the category information and position information of the candidate bounding box, the mask branch outputs the segmentation mask of the candidate bounding box, and determines the instance segmentation result of the remote sensing image according to the category information, position information and segmentation mask. Therefore, the present disclosure improves the accuracy and efficiency of obtaining the instance segmentation result of the remote sensing image.

[0103] The training process of the image instance segmentation model provided by the present disclosure is explained below.

[0104] like Figure 7 As shown, the training process of the image instance segmentation model proposed in this embodiment includes the following steps:

[0105] S701, obtaining training samples, wherein the training samples include labeled sample remote sensing images.

[0106] S702: Train the initial image instance segmentation model according to the training samples to obtain a trained image instance segmentation model.

[0107] In an embodiment of the present disclosure, after obtaining the training samples, the training samples are input into the initial image instance segmentation model to obtain the instance segmentation prediction results output by the initial image instance segmentation model. Based on the instance segmentation prediction results, the total loss function of the initial image instance segmentation model is obtained, wherein the total loss function includes a classification loss function, a bounding box regression sinusoidal loss function, and a mask loss function. According to the total loss function, the model parameters of the initial image instance segmentation model are updated to obtain a trained image instance segmentation model.

[0108] Among them, the total loss function L = L cls +L box +L mask , L cls is the classification loss function, L box is the bounding box regression sinusoidal loss function, L mask is the mask loss function.

[0109] Among them, the classification loss function L cls Using the cross entropy loss function, L mask The smoothL1 loss function is used.

[0110] It should be noted that, taking into account the problems of arbitrary orientation and multi-scale of objects in remote sensing images, the present invention usually adopts a rotated bounding box instead of a horizontal bounding box to realize the positioning of the instance. Different from the annotation form of the horizontal bounding box, the commonly used rotated bounding box annotation form is mainly divided into a five-parameter definition method and an eight-parameter definition method. The five-parameter definition method includes two methods: the OpenCV definition method and the long edge definition method.

[0111] Among them, the five-parameter definition method can be expressed as (x c ,y c ,w,h,θ), where (x c ,y c) refers to the center position of the rotated bounding box. In the long edge definition method, w and h are the shortest and longest edges in the rotated bounding box, respectively. The rotation angle θ ranges from [-90° to 90°).

[0112] It should be noted that since the OpenCV definition method has the problem of exchanging long sides and wide sides, and the eight-parameter definition method requires more parameters, the present disclosure adopts the long side definition method to represent the rotated bounding box. No matter which bounding box representation method is used, there is a boundary discontinuity problem, that is, the regression-based angle prediction method increases the loss sharply and cannot be continuous when the rotating box is at the boundary of the angle definition range, which makes the regression form of the model inconsistent in boundary and non-boundary conditions, thereby increasing the difficulty of model training and the difficulty of prediction under boundary conditions. In order to cope with the boundary discontinuity problem in the regression of the rotated bounding box, the present disclosure proposes a bounding box regression sinusoidal loss function, namely the sinusoidal loss function.

[0113] In the disclosed embodiments, the center position, shortest side, longest side and rotation angle of the rotated bounding box of the sample remote sensing image can be obtained from the training samples, and the predicted center position, predicted shortest side, predicted longest side and predicted rotation angle of the rotated bounding box of the sample remote sensing image can be obtained from the instance segmentation prediction results. The bounding box regression sinusoidal loss function is obtained based on the center position and the predicted center position, the shortest side and the predicted shortest side, the longest side and the predicted longest side, and the rotation angle and the predicted rotation angle.

[0114] It should be noted that in the long side definition method, Figure 8 As shown in the figure, the period of θ1-θ2 is π. When |θ1-θ2|<π / 2, the most ideal regression method is usually to reduce θ2 and regress in the counterclockwise direction. When |θ1-θ2|>π / 2, due to the periodicity of the angle, it is actually more desirable to increase θ2 and regress in the clockwise direction, thereby speeding up the regression. Therefore, |θ1-θ2| is constrained to In the interval, |sin(θ2-θ1)| takes π as the period, and the loss function becomes continuous at the boundary, which effectively solves the boundary discontinuity problem and will obtain a better convergence speed. At the same time, considering that θ2 will produce discontinuous vibrations at the periodic boundary, the bounding box regression sine loss function proposed in the present disclosure can be expressed as:

[0115] L box =|x c,2 -x c,1 |+|y c,2 -y c,1 |+|w2-w1|+|h2-h1|+|sin(θ2-θ1)|

[0116] Among them, L box is the bounding box loss function, (xc,2 ,y c,2 ) is the predicted center position, (x c,1 ,y c,1 ) is the center position, w2 is the predicted shortest side, w1 is the shortest side, h2 is the predicted longest side, h1 is the longest side, θ2 is the predicted rotation angle, and θ1 is the rotation angle.

[0117] In summary, the instance segmentation method of remote sensing images provided by the embodiments of the present disclosure obtains training samples, wherein the training samples include labeled sample remote sensing images, inputs the training samples into the initial image instance segmentation model, obtains the instance segmentation prediction results output by the initial image instance segmentation model, and obtains the total loss function of the initial image instance segmentation model based on the instance segmentation prediction results, wherein the total loss function includes the classification loss function, the bounding box regression sinusoidal loss function and the mask loss function. According to the total loss function, the model parameters of the initial image instance segmentation model are updated to obtain the trained image instance segmentation model. Therefore, the present disclosure trains the model through the bounding box loss function of the sinusoidal function, solves the boundary discontinuity problem in the rotation bounding box regression, improves the accuracy and reliability of the image instance segmentation model, and thus can more accurately and efficiently obtain the instance segmentation results of the remote sensing image.

[0118] The specific process of the instance segmentation method of remote sensing images proposed in the present invention is explained below.

[0119] For example, Fig. 9 As shown, a remote sensing image to be processed is obtained, and the remote sensing image is input into a feature extraction backbone network, that is, the remote sensing image is input into a block embedding layer to obtain an embedding vector sequence output by the block embedding layer, and the embedding vector sequence is input into multiple Mamba modules, and each Mamba module performs feature extraction on the embedding vector sequence to obtain a remote sensing image feature map of the remote sensing image, and the remote sensing image feature map is input into the Mask-RCNN in the image instance segmentation network, that is, the remote sensing image feature map is input into the RPN to obtain a candidate bounding box corresponding to the remote sensing image feature map output by the RPN, and feature extraction is performed on the candidate bounding box to obtain a target feature map of the same size, and the target feature map is respectively input into the classification regression branch and the mask branch, and the classification regression branch outputs the category information and position information of the candidate bounding box, and the mask branch outputs the segmentation mask of the candidate bounding box, and the instance segmentation result of the remote sensing image is determined according to the category information, position information and segmentation mask.

[0120] In summary, the instance segmentation method of the remote sensing image provided by the embodiment of the present disclosure constructs a feature extraction backbone network through the Mamba module, which can simultaneously have global situation modeling capabilities and linear time complexity, improves the accuracy and reliability of obtaining the feature map of the remote sensing image, and constructs an adaptive rotation variable scanning strategy in the feature extraction backbone network, which can adaptively learn the size, displacement and angle of the object according to the input, so as to adaptively learn the local area that needs to be scanned, and improve the feature extraction backbone network The ability to learn the features of objects with arbitrary directions and multi-scales solves the problems of arbitrary directions and multi-scales of objects in remote sensing images, and trains the model through the bounding box loss function of the sine function, solves the boundary discontinuity problem in the rotation bounding box regression, and improves the accuracy of the image instance segmentation model, thereby more accurately and efficiently obtaining the instance segmentation results of the remote sensing image.

[0121] Fig.10 FIG. 1 is a block diagram of a remote sensing image instance segmentation device according to an exemplary embodiment. Fig.10 As shown, the remote sensing image instance segmentation device 1000 according to the embodiment of the present disclosure may specifically include: an acquisition module 1001 , a first processing module 1002 , and a second processing module 1003 .

[0122] An acquisition module 1001 is used to acquire a remote sensing image to be processed;

[0123] A first processing module 1002 is used to input the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and a plurality of Mamba modules;

[0124] The second processing module 1003 is used to input the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network.

[0125] In one embodiment of the present disclosure, the first processing module 1002 is further used to: input the remote sensing image into the block embedding layer to obtain an embedding vector sequence output by the block embedding layer; input the embedding vector sequence into a first Mamba module among multiple Mamba modules, and the first Mamba module performs feature extraction on the embedding vector sequence to obtain a first remote sensing image feature map; and input the first remote sensing image feature map into a second Mamba module, and the second Mamba module continues to perform feature extraction on the first remote sensing image feature map to obtain a second remote sensing image feature map, and repeat the above process until the remote sensing image feature map is output by the last Mamba module.

[0126] In one embodiment of the present disclosure, each Mamba module includes a first linear layer, a second linear layer, a deep convolution layer, a first activation layer, a second activation layer, a multi-state space submodule, a Gate module, a third linear layer and a downsampling layer, and the first processing module 1002 is also used to: input the embedded vector sequence into the first linear layer and the first activation layer connected in sequence to obtain the first feature vector output by the first activation layer; input the embedded vector sequence into the second linear layer, the deep convolution layer and the second activation layer connected in sequence to obtain the second feature vector output by the activation layer; input the second feature vector into the multi-state space submodule to obtain the third feature vector output by each state space submodule, and input the third feature vector into the Gate module for fusion to obtain the fused feature vector output by the Gate module; perform residual connection on the first feature vector and the fused feature vector, and input the target feature vector after residual connection into the third linear layer and the downsampling layer to obtain the first remote sensing image feature map.

[0127] In one embodiment of the present disclosure, the first processing module 1002 is also used to: obtain a target scanning strategy associated with each state space sub-module, wherein the target scanning strategy includes a horizontal scanning strategy, a flipped horizontal scanning strategy, a vertical scanning strategy, a flipped vertical scanning strategy, a rotationally variable scanning strategy, and a flipped rotationally variable scanning strategy; and according to the target scanning strategy, obtain a third eigenvector output by each state space sub-module.

[0128] In one embodiment of the present disclosure, if the target scanning strategy is a rotation variable scanning strategy and a flip rotation variable scanning strategy, the first processing module 1002 is also used to: obtain a plurality of non-overlapping windows of fixed sizes pre-divided from the remote sensing image, and obtain a second eigenvector matching each of the non-overlapping windows; construct a routing function based on a global average pooling operation and a linear transformation operation, input the second eigenvector into the routing function, and predict the transformation parameters of each of the non-overlapping windows through the routing function, wherein the transformation parameters include scale, displacement and angle; according to the transformation parameters, transform each non-overlapping window into a rotation variable window, and within the rotation variable window, obtain the third eigenvector output by each state space submodule according to the rotation variable scanning strategy and the flip rotation variable scanning strategy.

[0129] In one embodiment of the present disclosure, the first processing module 1002 is further used to: obtain the weight corresponding to each third eigenvector; and perform weighted summation on each third eigenvector and the weight corresponding to each third eigenvector to obtain the fused eigenvector.

[0130] In one embodiment of the present disclosure, the second processing module 1003 is further used to: input the remote sensing image feature map into the region proposal network RPN to obtain a candidate bounding box corresponding to the remote sensing image feature map output by the RPN; perform feature extraction on the candidate bounding box to obtain a target feature map of the same size; input the target feature map into a classification regression branch and a mask branch respectively, the classification regression branch outputs the category information and position information of the candidate bounding box, and the mask branch outputs the segmentation mask of the candidate bounding box; determine the instance segmentation result of the remote sensing image based on the category information, the position information and the segmentation mask.

[0131] In one embodiment of the present disclosure, the training process of the image instance segmentation model includes: obtaining training samples, wherein the training samples include labeled sample remote sensing images; and training the initial image instance segmentation model according to the training samples to obtain a trained image instance segmentation model.

[0132] In one embodiment of the present disclosure, the training process of the image instance segmentation model includes: inputting the training samples into the initial image instance segmentation model to obtain the instance segmentation prediction results output by the initial image instance segmentation model; based on the instance segmentation prediction results, obtaining the total loss function of the initial image instance segmentation model, wherein the total loss function includes a classification loss function, a bounding box regression sinusoidal loss function and a mask loss function; according to the total loss function, updating the model parameters of the initial image instance segmentation model to obtain the trained image instance segmentation model.

[0133] In the embodiments of the present disclosure, the specific manner in which each module in the instance segmentation device for remote sensing images in the above-mentioned embodiments performs operations has been described in detail in the embodiments of the instance segmentation method for remote sensing images, and will not be repeated here.

[0134] In summary, the instance segmentation device of the remote sensing image provided by the embodiment of the present disclosure obtains the remote sensing image to be processed, inputs the remote sensing image into the feature extraction backbone network of the image instance segmentation model, so as to obtain the remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and multiple Mamba modules, and inputs the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model, so as to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network. Therefore, the present disclosure constructs a feature extraction backbone network through the block embedding layer and multiple Mamba modules, and the feature extraction backbone network can simultaneously have the global situation modeling capability and linear time complexity, thereby improving the accuracy and reliability of obtaining the remote sensing image feature map, solving the problem that the instance segmentation results in the remote sensing image have arbitrary directions and multi-scale objects, and improving the accuracy and efficiency of obtaining the instance segmentation results of the remote sensing image.

[0135] Fig.11 is a block diagram of an electronic device 1100 according to an exemplary embodiment.

[0136] like Fig.11 As shown, the electronic device 1100 includes:

[0137] The memory 1101 and the processor 1102, a bus 1103 connecting different components (including the memory 1101 and the processor 1102), the memory 1101 stores a computer program, and when the processor 1102 executes the program, the instance segmentation method of the remote sensing image of the embodiment of the present disclosure is implemented.

[0138] Bus 1103 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0139] The electronic device 1100 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 1100, including volatile and non-volatile media, removable and non-removable media.

[0140] The memory 1101 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1104 and / or cache memory 1105. The electronic device 1100 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 1106 may be used to read and write non-removable, non-volatile magnetic media ( Fig.11 not shown, usually called a "hard drive"). Although Fig.11 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 1103 via one or more data medium interfaces. The memory 1101 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0141] A program / utility 1108 having a set (at least one) of program modules 1107 may be stored, for example, in the memory 1101. Such program modules 1107 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 1107 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0142] The electronic device 1100 may also communicate with one or more external devices 1109 (e.g., keyboard, pointing device, display 1111, etc.), one or more devices that enable a user to interact with the electronic device 1100, and / or any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 1112. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1113. Fig.11 As shown, the network adapter 1113 communicates with other modules of the electronic device 1100 via the bus 1103. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] The processor 1102 executes the programs stored in the memory 1101 to perform various functional applications and data processing.

[0144] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the instance segmentation method of the remote sensing image of the embodiment of the present disclosure, and will not be repeated here.

[0145] The electronic device provided by the embodiment of the present disclosure obtains a remote sensing image to be processed, inputs the remote sensing image into a feature extraction backbone network of an image instance segmentation model, so as to obtain a remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and multiple Mamba modules, and inputs the remote sensing image feature map into an image instance segmentation network of an image instance segmentation model, so as to obtain an instance segmentation result of the remote sensing image output by the image instance segmentation network. Therefore, the present disclosure obtains the instance segmentation result of the remote sensing image through the image instance segmentation model, solves the problem that the instance segmentation result in the remote sensing image has arbitrary direction and multi-scale of the object, and improves the accuracy and efficiency of obtaining the instance segmentation result of the remote sensing image.

[0146] In order to implement the above embodiments, the present disclosure also proposes a computer-readable storage medium.

[0147] When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the instance segmentation method of the remote sensing image as described above. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0148] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0149] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for instance segmentation of remote sensing images, characterized in that: The method comprises: Acquiring remote sensing images to be processed; Inputting the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and a plurality of Mamba modules; The remote sensing image feature map is input into the image instance segmentation network of the image instance segmentation model to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network.

2. The method according to claim 1, characterized in that The step of inputting the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network comprises: Inputting the remote sensing image into the block embedding layer to obtain an embedding vector sequence output by the block embedding layer; The embedded vector sequence is input into a first Mamba module among multiple Mamba modules, and the first Mamba module performs feature extraction on the embedded vector sequence to obtain a first remote sensing image feature map, and the first remote sensing image feature map is input into a second Mamba module, and the second Mamba module continues to perform feature extraction on the first remote sensing image feature map to obtain a second remote sensing image feature map, and the above process is repeated until the last Mamba module outputs the remote sensing image feature map.

3. The method according to claim 2, characterized in that Each of the Mamba modules includes a first linear layer, a second linear layer, a deep convolution layer, a first activation layer, a second activation layer, a multi-state space submodule, a Gate module, a third linear layer and a downsampling layer. The first Mamba module performs feature extraction on the embedded vector sequence to obtain a first remote sensing image feature map, including: Inputting the embedding vector sequence into the first linear layer and the first activation layer connected sequentially to obtain a first feature vector output by the first activation layer; Inputting the embedding vector sequence into the second linear layer, the deep convolution layer and the second activation layer connected sequentially to obtain a second feature vector output by the activation layer; Inputting the second feature vector into the multi-state space submodule to obtain a third feature vector output by each state space submodule, and inputting the third feature vector into the Gate module for fusion to obtain a fused feature vector output by the Gate module; The first feature vector is residually connected with the fused feature vector, and the target feature vector after the residual connection is input into the third linear layer and the downsampling layer to obtain the first remote sensing image feature map.

4. The method according to claim 3, characterized in that The step of inputting the second feature vector into the multi-state space submodule to obtain a third feature vector output by each state space submodule includes: Obtaining a target scanning strategy associated with each state space submodule, wherein the target scanning strategy includes a horizontal scanning strategy, a flipped horizontal scanning strategy, a vertical scanning strategy, a flipped vertical scanning strategy, a rotationally variable scanning strategy, and a flipped rotationally variable scanning strategy; According to the target scanning strategy, the third eigenvector output by each state space submodule is obtained.

5. The method according to claim 4, characterized in that If the target scanning strategy is a rotation variable scanning strategy and a flip rotation variable scanning strategy, obtaining the fourth image feature vector output by each state space submodule according to the target scanning strategy includes: Acquire a plurality of non-overlapping windows of fixed sizes pre-divided from the remote sensing image, and acquire a second feature vector matching each of the non-overlapping windows; Constructing a routing function according to a global average pooling operation and a linear transformation operation, inputting the second feature vector into the routing function, and predicting the transformation parameters of each of the non-overlapping windows through the routing function, wherein the transformation parameters include scale, displacement and angle; According to the transformation parameters, each non-overlapping window is transformed into a rotationally variable window, and within the rotationally variable window, the third eigenvector output by each state space submodule is obtained according to the rotationally variable scanning strategy and the flip rotationally variable scanning strategy.

6. The method according to claim 3, characterized in that The step of inputting the third feature vector into the Gate module for fusion to obtain a fused feature vector output by the Gate module includes: Get the weight corresponding to each third eigenvector; A weighted sum is performed on each of the third eigenvectors and the weight corresponding to each of the third eigenvectors to obtain the fused eigenvector.

7. The method according to claim 1, characterized in that The step of inputting the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network comprises: Inputting the remote sensing image feature map into a region proposal network (RPN) to obtain a candidate bounding box corresponding to the remote sensing image feature map output by the RPN; Performing feature extraction on the candidate bounding box to obtain a target feature map of the same size; The target feature map is input into the classification and regression branch and the mask branch respectively, the classification and regression branch outputs the category information and position information of the candidate bounding box, and the mask branch outputs the segmentation mask of the candidate bounding box; An instance segmentation result of the remote sensing image is determined according to the category information, the location information and the segmentation mask.

8. The method according to any one of claims 1 to 7, characterized in that The training process of the image instance segmentation model includes: Acquire training samples, wherein the training samples include labeled sample remote sensing images; The initial image instance segmentation model is trained according to the training samples to obtain a trained image instance segmentation model.

9. The method according to claim 8, characterized in that The step of training the initial image instance segmentation model according to the training samples to obtain a trained image instance segmentation model includes: Inputting the training sample into an initial image instance segmentation model to obtain an instance segmentation prediction result output by the initial image instance segmentation model; Based on the instance segmentation prediction result, obtaining a total loss function of the initial image instance segmentation model, wherein the total loss function includes a classification loss function, a bounding box regression sinusoidal loss function, and a mask loss function; According to the total loss function, the model parameters of the initial image instance segmentation model are updated to obtain the trained image instance segmentation model.

10. The method according to claim 9, characterized in that The obtaining of the bounding box regression sinusoidal loss function comprises: Obtaining, from the training sample, a center position, a shortest side, a longest side, and a rotation angle of a rotation bounding box of the sample remote sensing image; Obtaining a predicted center position, a predicted shortest side, a predicted longest side, and a predicted rotation angle of a rotation bounding box of the sample remote sensing image from the instance segmentation prediction result; The bounding box regression sinusoidal loss function is obtained according to the center position and the predicted center position, the shortest side and the predicted shortest side, the longest side and the predicted longest side, and the rotation angle and the predicted rotation angle.

11. A remote sensing image instance segmentation device, characterized in that: The device comprises: An acquisition module, used for acquiring remote sensing images to be processed; A first processing module is used to input the remote sensing image into a feature extraction backbone network of an image instance segmentation model to obtain a remote sensing image feature map output by the feature extraction backbone network, wherein the feature extraction backbone network includes a block embedding layer and a plurality of Mamba modules; The second processing module is used to input the remote sensing image feature map into the image instance segmentation network of the image instance segmentation model to obtain the instance segmentation result of the remote sensing image output by the image instance segmentation network.

12. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: in, The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images