Image segmentation method and device

Abstract

The invention provides an image segmentation method and device, and the method comprises the steps: obtaining satellite data containing a space target, and constructing a data set; determining a training sample set from the data set; inputting the training sample into a space target segmentation network model, wherein the space target segmentation network model comprises a Unet network model, a first convolutional layer and a second convolutional layer, wherein the first convolutional layer and the second convolutional layer are connected with the output of the Unet network model and are parallel to each other; the space target segmentation network model further comprises an edge network model, the edge sensing network model is used for acquiring image features extracted by the encoder, and the edge sensing network model is used for acquiring a binary edge prediction image and binary edge loss from the image features; and obtaining a to-be-segmented satellite image containing the space target, and inputting the to-be-segmented satellite image containing the space target into the trained space target segmentation network model to obtain a segmentation result of the to-be-segmented satellite image containing the space target. According to the method, the accuracy and robustness of a segmentation result are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an image segmentation method and apparatus. Background Technology

[0002] In the field of modern computer vision, spatial object component segmentation is a challenging task. This task aims to accurately extract objects from complex backgrounds and further precisely segment the individual components within the object. This goes beyond simply separating the object as a whole from the background; it requires delving into the object's internal structure, separating its individual components to gain an understanding and description of the object's complete structure. This precise segmentation includes not only the main parts of the object but also its fine internal structures and edges to ensure the segmentation results closely resemble reality, thus meeting the requirement for high accuracy.

[0003] However, satellite imagery of space targets presents numerous challenges, including low resolution, uneven illumination, and significant changes in viewing angle, which further complicates component segmentation. Traditional image segmentation methods are typically based on pixel-level features such as color, texture, and shape. However, these methods generally perform poorly when faced with the aforementioned difficulties, making the design of a sophisticated method for fine-grained component segmentation of space targets an urgent task.

[0004] In recent years, the development of deep learning technology has brought new hope to spatial object segmentation. In particular, methods based on deep neural networks (DNNs) have made significant progress in image segmentation tasks. These methods learn representations of object parts from large amounts of labeled data through end-to-end learning, thereby achieving more accurate segmentation results. However, when applying deep learning to spatial object segmentation, it often only considers the segmentation of the object as a whole in the spatial context, while lacking the analysis and utilization of the object's structure. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes an image segmentation method and apparatus.

[0006] According to a first aspect of the present invention, an image segmentation method is provided, the method comprising the following steps:

[0007] Step S1: Acquire satellite data containing space targets and construct a dataset; determine a training sample set from the dataset;

[0008] Step S2: Input the training samples into the spatial target segmentation network model. The spatial target segmentation network model includes a Unet network model, a first convolutional layer connected to the output of the Unet network model, and a second convolutional layer. The Unet network model includes an encoder and a decoder. The encoder is used to extract image features from the training samples, and the decoder generates an image feature map based on the image features. The first convolutional layer generates a binary classification prediction result based on the image feature map, and the second convolutional layer generates a ternary classification prediction result based on the image feature map. The spatial target segmentation network model also includes an edge network model. The edge-aware network model is used to obtain the image features extracted by the encoder, and the edge-aware network model is used to obtain a binary edge prediction image and a binary edge loss from the image features. Based on the binary edge prediction image, the binary classification prediction result, and the ternary classification prediction result, a segmentation result is obtained. Based on the ternary classification prediction result, a symmetric edge loss is determined. Based on the binary classification prediction result and the ternary classification prediction result, a cross-entropy loss is determined. Based on the symmetric edge loss, the cross-entropy loss, and the binary edge loss, a loss function is constructed, and the spatial target segmentation network model is trained to obtain the trained spatial target segmentation network model.

[0009] Step S3: Obtain the satellite image containing space targets to be segmented, input it into the trained space target segmentation network model, and obtain the segmentation result of the satellite image containing space targets to be segmented.

[0010] Preferably, the encoder includes a first convolutional block, a second convolutional block, a third convolutional block, a CBAM module, a fourth convolutional module, a SENet module, and a fifth convolutional block connected in sequence. Each convolutional block includes several convolutional layers and a max pooling layer. Each convolutional block generates a feature layer, denoted as C1, C2, C3, C4, and C5, respectively. The C5 generated by the fifth convolutional block is input into the decoder.

[0011] Preferably, the binary classification prediction result includes segmenting the main body and solar panel portion of the space target; the ternary classification prediction result includes segmenting the main body, solar panel, and antenna portion of the space target.

[0012] Preferably, the edge-aware network model includes three parallel branches, each branch including a convolutional layer and a channel attention module. The three parallel branches are respectively input to the outputs C2, C3, and C4 of the second, third, and fourth convolutional modules. The outputs C'2, C'3, and C'4 of the three branches are all input to a convolutional layer for concatenating the three outputs. The concatenated result is mapped to the edge detection result space to obtain a binary edge prediction image. Based on the binary edge prediction image, the binary edge loss is determined.

[0013] Preferably, the loss function is:

[0014] Loss = γ * [loss ce (I,L)+loss symmetric [(I, α)]+(1-γ)*loss edge (I edge L edge )

[0015] Where I represents the segmentation prediction result for three-class classification or two-class classification. edge For binary edge prediction images, L edge Here, I represents the binary edge image label, γ is the weight coefficient, L is the three-class segmentation mask label map corresponding to I or the two-class segmentation mask label map, α is the weight coefficient for calculating the symmetry loss, and loss... edge (I edge L edge ) represents the binary marginal loss, loss symmetric (I, α) represents the symmetric edge loss. ce (I, L) represents the cross-entropy loss between the three-class or two-class segmentation prediction result and the corresponding three-class or two-class segmentation mask label map.

[0016] Preferably,

[0017] Where α is the weighting coefficient for calculating the symmetry loss, flip_img is the image of the training sample that has been horizontally or vertically flipped, and total_pixel is the total number of pixels in I.

[0018] α=max(similarity_horizon,similarity_vertic al)

[0019]

[0020]

[0021] Where similarity_horizon represents the measure of horizontal symmetry, similarity_vertical represents the measure of vertical symmetry, flip_img1 is the image after horizontally flipping the training sample, and flip_img2 is the image after vertically flipping the training sample.

[0022] Preferably, loss edge (I edge L edge =CrossEntropy(Iedge L edge (weights)

[0023] weights=(w neg w pos )

[0024]

[0025]

[0026] Among them, w pos w represents the weight of the positive sample. neg The weights represent the negative samples, neg_num represents the number of negative samples, pos_num represents the number of positive samples, weights represents the sample weight vector, and target represents the target edge image, i.e., the true edge information. edge For binary edge prediction images, CrossEntropy is the cross-entropy loss function used to measure the difference between the predicted edge image and the true edge information.

[0027] According to a second aspect of the present invention, an image segmentation apparatus is provided, the apparatus comprising:

[0028] Training sample set generation module: configured to acquire satellite data containing space targets, construct a dataset, and determine a training sample set from the dataset;

[0029] Training module: Configured to input training samples into a spatial object segmentation network model. The spatial object segmentation network model includes a Unet network model, a first convolutional layer connected to the output of the Unet network model, and a second convolutional layer. The Unet network model includes an encoder and a decoder. The encoder extracts image features from the training samples, and the decoder generates an image feature map based on these features. The first convolutional layer generates a binary classification prediction result based on the image feature map, and the second convolutional layer generates a tri-class classification prediction result based on the image feature map. The spatial object segmentation network model also includes an edge network model. The edge-aware network model is used to acquire image features extracted by the encoder. It is also used to obtain a binary edge prediction image and a binary edge loss from these image features. Based on the binary edge prediction image, the binary classification prediction result, and the ternary classification prediction result, a segmentation result is obtained. A symmetric edge loss is determined based on the ternary classification prediction result. A cross-entropy loss is determined based on the binary and ternary classification prediction results. A loss function is constructed based on the symmetric edge loss, cross-entropy loss, and binary edge loss. The spatial target segmentation network model is then trained to obtain the trained spatial target segmentation network model.

[0030] Segmentation module: configured to acquire a satellite image containing space targets to be segmented, input it into a trained space target segmentation network model, and obtain the segmentation result of the satellite image containing space targets to be segmented.

[0031] According to a third aspect of the present invention, an electronic device is provided, comprising:

[0032] A processor is used to execute multiple instructions;

[0033] Memory, used to store multiple instructions;

[0034] The plurality of instructions are to be stored in the memory and loaded and executed by the processor as described above.

[0035] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein a plurality of instructions are stored therein; the plurality of instructions are configured to be loaded by a processor and executed as described above.

[0036] This invention proposes an image segmentation method and apparatus for segmenting spatial target components. The method considers the structural characteristics of spatial targets and proposes a symmetry loss method that utilizes symmetry features present on the targets to constrain the model's segmentation of target components. An edge-aided network is then added to better perceive the boundaries between components. Finally, the cross-entropy loss, edge loss, and symmetry loss are combined to comprehensively constrain the component segmentation process. This approach aims to overcome the limitations of traditional methods, effectively utilize the symmetry information of target components, improve the segmentation network's ability to perceive edge details, and achieve more accurate and robust segmentation of spatial target components.

[0037] The image segmentation method and apparatus of the present invention utilize the UNet network structure to simultaneously perform two segmentation tasks, and are supplemented by an edge-aware network. The symmetry loss, binary edge loss and cross loss are combined and weighted to improve the accuracy and robustness of the segmentation results.

[0038] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0039] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention, and the invention is illustrated by the following drawings. In the drawings:

[0040] Figure 1 This is a schematic flowchart of an image segmentation method according to one embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the image segmentation method architecture according to one embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of a dataset construction process according to one embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of an edge-aware network structure according to one embodiment of the present invention;

[0044] Figure 5 This is a structural block diagram of an image segmentation device according to one embodiment of the present invention. Detailed Implementation

[0045] First, combine Figure 1 This invention describes an image segmentation method according to one embodiment of the present invention. For example... Figures 1-2 As shown, the method includes the following steps:

[0046] Step S1: Acquire satellite data containing space targets and construct a dataset; determine a training sample set from the dataset;

[0047] Step S2: Input the training samples into the spatial target segmentation network model. The spatial target segmentation network model includes a Unet network model, a first convolutional layer connected to the output of the Unet network model, and a second convolutional layer. The Unet network model includes an encoder and a decoder. The encoder is used to extract image features from the training samples, and the decoder generates an image feature map based on the image features. The first convolutional layer generates a binary classification prediction result based on the image feature map, and the second convolutional layer generates a ternary classification prediction result based on the image feature map. The spatial target segmentation network model also includes an edge network model. The edge-aware network model is used to obtain the image features extracted by the encoder, and the edge-aware network model is used to obtain a binary edge prediction image and a binary edge loss from the image features. Based on the binary edge prediction image, the binary classification prediction result, and the ternary classification prediction result, a segmentation result is obtained. Based on the ternary classification prediction result, a symmetric edge loss is determined. Based on the binary classification prediction result and the ternary classification prediction result, a cross-entropy loss is determined. Based on the symmetric edge loss, the cross-entropy loss, and the binary edge loss, a loss function is constructed, and the spatial target segmentation network model is trained to obtain the trained spatial target segmentation network model.

[0048] Step S3: Obtain the satellite image containing space targets to be segmented, input it into the trained space target segmentation network model, and obtain the segmentation result of the satellite image containing space targets to be segmented.

[0049] like Figure 3As shown, step S1 involves: acquiring satellite data containing space targets and constructing a dataset; determining a training sample set from the dataset, wherein:

[0050] Since space target data typically originates from remote sensing platforms such as satellites and drones, acquiring this data may require expensive equipment and technology, as well as collaboration with relevant institutions. This increases the difficulty and cost of data acquisition, and the number of publicly available space target datasets is limited compared to datasets in other fields, especially datasets specific to particular tasks and scenarios. All datasets used in this invention are simulation datasets obtained by collecting publicly available 3D satellite models. To provide stronger data support for space target recognition algorithm research, and considering factors such as satellite lighting issues, motion blur, and multi-view variations, a space target simulation dataset more consistent with the actual project was created, based on the background requirements of the cross-view space target recognition project. This dataset generates a new dataset for the cross-view target recognition project by using two methods: multi-angle fixed lighting rendering of 3D satellite models and combining textural images with multi-view diffusion.

[0051] This invention uses space target images containing 20 satellite categories, totaling 418 images as the training and validation sets for the model, and then uses ten types of space target images that did not appear in the training and validation sets as the test set.

[0052] Based on the traditional U-Net network model, this model employs an encoder-decoder architecture. The encoder consists of four blocks, each with two convolutional layers and one max-pooling layer. It downsamples the input image and extracts high-level features. This process is typically implemented using convolutional and pooling layers. In the U-Net model, the encoder's task is to transform the input image into a smaller feature representation while preserving important semantic information. Each downsampling operation reduces the image size but increases the number of features, which helps capture global information and abstract features of the image. The decoder also consists of four blocks, each with one upsampling layer and two convolutional layers. It is responsible for converting the feature maps extracted by the encoder back to the original input image size and using them for pixel-level classification. The decoder typically includes upsampling operations to progressively restore the spatial resolution of the image. In U-Net, the decoder's task is to receive the encoder's feature maps and combine them into the original-size feature map. Each step in the decoder fuses low-resolution features with high-resolution features through upsampling and stitching operations to recover detail and spatial information. The input image is then processed by the encoder-decoder to obtain the encoded-decoded result.

[0053] In this invention, given the significant variations in the attitude of space targets and the high degree of similarity between different components from certain perspectives, making them difficult to distinguish, research has revealed that solar panels and antennas exhibit high structural similarity from certain perspectives, posing a challenge to their differentiation. This similarity is not only reflected in texture, color, and structure but also in their shape and features. Therefore, traditional methods based on texture, color, and structure information often fail in such cases, and deep network segmentation also yields poor results. Thus, the encoder and decoder were improved, and a multi-branch model was implemented at the output of the Unet network model, with parallel first and second convolutional layers connected to the output. For components that are difficult to distinguish, they are divided into two tasks. This dual-task approach increases the model's robustness. When one task performs poorly in certain situations, the other task can compensate for its shortcomings, thereby improving overall performance. This multi-task learning method makes the model more robust and capable of effectively classifying under different conditions.

[0054] The encoder includes a first convolutional block, a second convolutional block, a third convolutional block, a CBAM module, a fourth convolutional module, a SENet module, and a fifth convolutional block connected in sequence. Each convolutional block includes several convolutional layers and a max pooling layer. Each convolutional block generates a feature layer, denoted as C1, C2, C3, C4, and C5, respectively. The C5 generated by the fifth convolutional block is input into the decoder.

[0055] In this invention, C5 performs an upsampling operation through the decoder to gradually restore the resolution of the feature map, and finally inputs it into the convolutional layers for three-class classification and two-class classification respectively to obtain the segmentation prediction results for three-class classification and two-class classification.

[0056] Furthermore, the binary classification prediction result includes segmenting the main body and solar panel portion of the space target; the ternary classification prediction result includes segmenting the main body, solar panel, and antenna portion of the space target. The binary classification prediction result is used when the antenna portion of the target is difficult to identify due to limitations in viewing angle and resolution. The ternary classification prediction result is used when it is necessary to distinguish the different components of the target in greater detail, such as in performing more refined structural analysis, or when the viewing angle and resolution are good enough to distinguish different target parts.

[0057] The edge-aware network model is used to enhance the perception of target edge information. The feature layers (C1, C2, C3, C4, C5) extracted by the encoder of the edge-aware network model extract information from feature maps at different scales, which are typically derived from network layers. This helps capture semantic information at different scales, improving the model's perception capability. Finally, a feature concatenation operation concatenates the feature maps at different scales to form a richer feature representation. This helps improve the model's ability to capture target boundaries and details, thereby improving the accuracy of semantic segmentation.

[0058] Furthermore, the edge-aware network model includes three parallel branches, each including a convolutional layer and a channel attention module. The three parallel branches are respectively input to the outputs C2, C3, and C4 of the second, third, and fourth convolutional modules. The outputs C′2, C′3, and C′4 of the three branches are all input to a convolutional layer used to concatenate the three outputs. The concatenated result is mapped to the edge detection result space to obtain a binary edge prediction image. Based on the binary edge prediction image, the binary edge loss is determined.

[0059] In this invention, the edge-aware network module structure includes multiple convolutional layers and channel attention modules. By integrating feature information at different scales, it effectively achieves accurate edge detection of images. First, the three feature layers (C2, C3, C4) obtained from the encoder are input into the edge network, and feature information at different scales is extracted by the three convolutional layers respectively. Next, the fused feature map is upsampled to make its size the same as the original image for feature fusion. Finally, the fused feature map is mapped to the edge detection result space through convolutional layers, and a binary edge map is output.

[0060] The process of obtaining segmentation results based on the binary edge prediction image, binary classification prediction results, and ternary classification prediction results includes: using the binary edge prediction image as the edges of each region of the image segmentation, and using the binary classification prediction results or ternary classification prediction results as component identifiers of spatial targets, thereby obtaining the segmentation status of each region.

[0061] The loss function is:

[0062] Loss = γ * [loss ce (I,L)+loss symmetric [(I, α)]+(1-γ)*loss edge (I edge L edge )

[0063] Where I represents the segmentation prediction result for three-class classification or two-class classification. edgeFor binary edge prediction images, L edge Here, I represents the binary edge image label, γ is the weight coefficient, L is the three-class segmentation mask label map corresponding to I or the two-class segmentation mask label map, α is the weight coefficient for calculating the symmetry loss, and loss... edge (I edge L edge ) represents the binary marginal loss, loss symmetric (I, α) represents the symmetric edge loss. ce (I, L) represents the cross-entropy loss between the three-class or two-class segmentation prediction result and the corresponding three-class or two-class segmentation mask label map.

[0064]

[0065] Where α is the weighting coefficient for calculating the symmetry loss, flip_img is the image of the training sample that has been horizontally or vertically flipped, and total_pixel is the total number of pixels in I.

[0066] α=max(similarity_horizon,similarity_vertic al)

[0067]

[0068]

[0069] Where similarity_horizon represents the measure of horizontal symmetry, similarity_vertical represents the measure of vertical symmetry, flip_img1 is the image after horizontally flipping the training sample, and flip_img2 is the image after vertically flipping the training sample.

[0070] loss edge (I edge L edge =CrossEntropy(I edge L edge (weights)

[0071] weights=(w neg w pos )

[0072]

[0073]

[0074] Among them, w pos w represents the weight of the positive sample.neg The weights represent the negative samples, neg_num represents the number of negative samples, pos_num represents the number of positive samples, weights represents the sample weight vector, and target represents the target edge image, i.e., the true edge information. edge For binary edge prediction images, CrossEntropy is the cross-entropy loss function used to measure the difference between the predicted edge image and the true edge information.

[0075] This invention proposes a novel loss function that combines symmetry loss, binary edge loss, and cross loss to balance the relationship between two segmentation tasks and guide the network model to learn more accurate and robust segmentation results. Specifically, symmetry loss preserves the symmetry between target components, binary edge loss improves edge sharpness, and cross loss performs global optimization of the segmentation results.

[0076] The loss function of this invention effectively considers multiple factors such as image edge detection, symmetry preservation, and classification accuracy. The binary loss function is mainly used to measure the difference between the predicted edge map and the true edge map. Through the weighted binary loss function, this invention allows the model to focus more on edge accuracy, that is, to better capture the boundary information of objects in the image. This ensures that the segmented parts have clear boundaries, helping to accurately distinguish the boundaries between different parts. The symmetry loss function is used to maintain the symmetry of the image in the horizontal and vertical directions. Through the weighted symmetry loss function, this invention can encourage the model to pay more attention to the symmetry of the image, guiding the model to classify pixels in symmetrical parts into the same category as much as possible, while also suppressing unnatural part shapes. The cross-entropy loss function is used to measure the difference between the class probability distribution output by the model and the true category. Through the weighted cross-entropy loss function, this invention allows the model to focus more on the classification accuracy of different parts in the image, ensuring that each part is correctly classified. The comprehensive loss function is used for binary edge loss. edge ) and symmetry regularization loss symmetric and cross-entropy loss ce We use weighted summation to constrain the model during training.

[0077] This invention provides a method for calculating loss. symmetric A specific embodiment of (I, α).

[0078] Symmetric regularized loss symmetricThis method is designed based on the symmetry features observed in satellite target images. Space targets in three-dimensional space often possess a certain degree of symmetry, but this symmetry can be lost from certain perspectives in two-dimensional images. Therefore, a method is proposed to determine whether symmetry exists in current space target images, classifying all targets into three categories: those with horizontal symmetry, those with vertical symmetry, and those without symmetry. This method can better understand and utilize the symmetry features of targets under different perspectives, thereby helping to improve the accuracy and stability of space target component segmentation.

[0079] Target detection is performed on the target image. The target is extracted, and all pixels in the label image are traversed. Each pixel is analyzed to determine if it is a target pixel, and the position of each pixel is recorded. The positions of the four boundary points are obtained, which are X, X, and X'. min X max Y min Y max Then, a rectangular box is drawn on the image to mark the target area.

[0080] This invention relates to a method for image symmetry detection and its application. It measures the symmetry of the detected target regions using a simple method. Euclidean distance is used to measure differences in the horizontal or vertical direction of the image, and then the horizontal or vertical mirror similarity of the image is calculated. The specific steps are as follows:

[0081] Euclidean distance is used to measure the degree of difference. Euclidean distance is the distance between two points in a vector space, and the formula is:

[0082]

[0083] Where I is the original image, flip_img is the horizontal mirror image of the image, and ∑ represents the sum of the pixels. This Euclidean distance reflects the degree of difference between each pixel in the horizontal direction of the image.

[0084] The Euclidean distance is converted into a similarity metric to assess the symmetry of the images. The conversion is performed using the following formula:

[0085]

[0086] Here, 255 represents the range of pixel values. Since a smaller Euclidean distance indicates a more similar image, this invention divides the distance by 255 to normalize it to the range between 0 and 1, and then subtracts 1 to obtain the similarity. The closer the value is to 1, the more symmetrical the image is, thus obtaining the mirror similarity of the image in the horizontal or vertical direction, which is used to evaluate the horizontal symmetry of the image.

[0087] By further comparing the horizontal and vertical approximations on the image, the possible symmetry types of the image are determined. A symmetry loss is then calculated, aiming to classify pixels in symmetrical parts into the same category. The proposed symmetry loss constrains the model, classifying pixels in symmetrical parts into the same category.

[0088] α=max(similarity_horizon, similarity_vertical)

[0089] A regularized loss method for adjusting the symmetry of neural network models effectively adjusts the symmetry of the model during segmentation network training, helping to improve the model's performance and stability. After the above similarity calculation, it is determined whether the value is greater than the set symmetry similarity threshold. If it is greater, the symmetry loss is calculated, and the calculation formula is as follows:

[0090]

[0091] This invention provides a method for calculating loss. edge (I edge L edge One specific embodiment of ).

[0092] Calculate binary marginal loss edge After obtaining the predicted binary edge map, binary edge loss is used to constrain the image boundaries. By analyzing the difference between the target edge and the predicted edge, and combining the weight adjustment of positive and negative samples, the image edge detection task is effectively optimized. The binary edge loss calculation method includes the following steps:

[0093] The number of positive and negative samples in the target image is calculated to determine the sample weights, using the following formula:

[0094]

[0095]

[0096] The weights of positive and negative samples are calculated based on the number of samples to balance the contributions of positive and negative samples in the loss function.

[0097] weights=(w neg w pos )

[0098] The predicted binary edge map and the target image are subjected to cross-entropy loss calculation, taking into account sample weights.

[0099] loss edge (I edge L edge =CrossEntropy(Iedge L edge (weights)

[0100] Among them, w pos The weights of positive samples represent the proportion of positive samples in the loss function, w. neg `neg_num` represents the weight of the negative samples, indicating their proportion of contribution to the loss function. `pos_num` represents the number of negative samples in the target image. `weights` represents the sample weight vector, including the weights of positive and negative samples. edge This represents the predicted binary edge map after interpolation. L edge This represents the target edge map, i.e., the actual edge information.

[0101] The training method of this invention first performs image segmentation using an improved U-Net network structure, leveraging an edge-aware network to enhance edge information extraction. Next, the model is trained using a designed comprehensive loss function. This method employs a step-by-step training strategy: initially freezing the backbone network and training only the decoder; later unfreezing the backbone network for end-to-end full network training to adapt to different training stages. This strategy effectively avoids gradient vanishing and exploding problems, improving training stability and convergence speed. Furthermore, this invention provides support for multi-GPU distributed training and a method for adaptively adjusting the learning rate based on the current batch size.

[0102] The experimental results of this invention are as follows.

[0103] Results of the 3-class classification experiment: In the 3-class classification task, using the original multi-branch U-Net model, the MIOU reached 63.63% and MPA reached 68.54%. After adding an edge detection network, the MIOU improved to 64.56% and the MPA improved to 69.73%. Further increasing the symmetry loss, the experimental results show that in the 3-class classification task, the MIOU reached 65.3% and the MPA reached 70.27%.

[0104] Model antenna Solar panels main body MIOU MPA Unet 0.07 0.78 0.7 63.63％ 68.54％ Unet_edge 0.09 0.8 0.72 64.56％ 69.73％ Unet_edge_symmetric 0.09 0.81 0.73 65.3％ 70.27％

[0105] Results of binary classification experiments: In the binary classification task, after using the original UET, the MIOU reached 83.04% and the MPA reached 88.9%. By adding an edge detection network to the original model and fusing edge information into the segmentation results, the MIOU increased to 83.06% and the MPA increased to 89.02%. After further adding symmetry loss, the MIOU increased to 83.06% and the MPA increased to 89.61%.

[0106] Model Solar panels main body MIOU MPA Unet 0.7 0.8 83.04％ 88.9％ Unet_edge 0.82 0.72 83.06％ 89.02％ Unet_edge_symmetric 0.84 0.73 84.82％ 89.61％

[0107] These results demonstrate that introducing an edge detection network and symmetry loss into the original U-Net model significantly improves its performance in both three-class and two-class classification tasks. In the three-class classification task, from the initial U-Net to the addition of an edge detection network and then to the addition of symmetry loss, MIOU and MPA show a gradual improvement trend, reaching 65.3% and 70.27%, respectively. In the two-class classification task, similar improvements also resulted in higher MIOU and MPA for both solar panels and the main body. In summary, these experimental results demonstrate the effectiveness of improving segmentation models by introducing edge information and symmetry loss constraints, and show that these methods can achieve performance improvements across different types of segmentation tasks.

[0108] When performing symmetry measurement, this invention sets a symmetry evaluation threshold. The quality of the threshold will affect the segmentation effect of the model. Based on this, different symmetry evaluation thresholds are tried in the experiment. Three thresholds are tested respectively. Experiments are conducted according to different symmetry evaluation thresholds, and the segmentation results of the image segmentation results in the three parts of the antenna, solar panel and main body are observed.

[0109] Results of the three-class classification experiment: At a threshold of 0.7, the segmentation accuracy for each part is as follows: antenna 0.08, solar panel 0.78, and main body 0.71. As the threshold increases to 0.75, the segmentation accuracy for the antenna, solar panel, and main body improves to 0.09, 0.81, and 0.73, respectively. At a threshold of 0.8, the segmentation accuracy for the antenna, solar panel, and main body is 0.09, 0.78, and 0.7, respectively.

[0110] Regarding segmentation metrics, MIOU also showed an upward trend as the threshold increased, rising from 64.22% to 65.3%, but slightly decreasing to 64.3% at the 0.8 threshold. MPA reached its highest value of 70.27% at the 0.75 threshold, while MPA remained relatively stable at other thresholds, at 69.93% and 69.88%, respectively.

[0111] threshold antenna Solar panels main body MIOU MPA 0.7 0.08 0.78 0.71 64.22％ 69.93％ 0.75 0.09 0.81 0.73 65.3％ 70.27％ 0.8 0.09 0.78 0.7 64.3％ 69.88％

[0112] Results of the binary classification experiment: Based on the results of the binary classification experiment, the segmentation of the solar panel and the main body was observed under different symmetry evaluation thresholds. At a threshold of 0.7, the segmentation accuracy of the solar panel was 0.8, and the segmentation accuracy of the main body was 0.71. As the threshold increased to 0.75, the segmentation accuracy of the solar panel and the main body improved to 0.84 and 0.73, respectively. At a threshold of 0.8, the segmentation accuracy of the solar panel and the main body were 0.79 and 0.69, respectively.

[0113] Regarding segmentation metrics, MIOU showed a gradual increasing trend as the threshold increased, rising from 83.41% to 84.82%, but slightly decreasing to 83% at the 0.8 threshold. MPA reached its highest value at the 0.75 threshold, at 89.61%, while MPA remained relatively stable at other thresholds, at 89.39% and 89.13%, respectively.

[0114] threshold Solar panels main body MIOU MPA 0.7 0.8 0.71 83.41％ 89.39％ 0.75 0.84 0.73 84.82％ 89.61％ 0.8 0.79 0.69 83％ 89.13％

[0115] This invention provides an image segmentation apparatus, the apparatus comprising:

[0116] Training sample set generation module: configured to acquire satellite data containing space targets, construct a dataset, and determine a training sample set from the dataset;

[0117] Training module: Configured to input training samples into a spatial object segmentation network model. The spatial object segmentation network model includes a Unet network model, a first convolutional layer connected to the output of the Unet network model, and a second convolutional layer. The Unet network model includes an encoder and a decoder. The encoder extracts image features from the training samples, and the decoder generates an image feature map based on these features. The first convolutional layer generates a binary classification prediction result based on the image feature map, and the second convolutional layer generates a tri-class classification prediction result based on the image feature map. The spatial object segmentation network model also includes an edge network model. The edge-aware network model is used to acquire image features extracted by the encoder. It is also used to obtain a binary edge prediction image and a binary edge loss from these image features. Based on the binary edge prediction image, the binary classification prediction result, and the ternary classification prediction result, a segmentation result is obtained. A symmetric edge loss is determined based on the ternary classification prediction result. A cross-entropy loss is determined based on the binary and ternary classification prediction results. A loss function is constructed based on the symmetric edge loss, cross-entropy loss, and binary edge loss. The spatial target segmentation network model is then trained to obtain the trained spatial target segmentation network model.

[0118] Segmentation module: configured to acquire a satellite image containing space targets to be segmented, input it into a trained space target segmentation network model, and obtain the segmentation result of the satellite image containing space targets to be segmented.

[0119] The present invention further provides an electronic device, comprising:

[0120] A processor is used to execute multiple instructions;

[0121] Memory, used to store multiple instructions;

[0122] The plurality of instructions are to be stored in the memory and loaded and executed by the processor as described above.

[0123] The present invention further provides a computer-readable storage medium storing a plurality of instructions; the plurality of instructions are used by a processor to load and execute the method as described above.

[0124] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0125] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0128] The integrated unit, implemented as a software functional unit, can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, a physical server, or a cloud server, etc., requiring the Ubuntu operating system to be installed) to execute some steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0129] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method of image segmentation, characterized in that, The method comprises the following steps: Step S1: acquiring satellite data containing space targets, and constructing a data set; determining a training sample set from the data set; Step S2: inputting the training sample into a space target segmentation network model, the space target segmentation network model comprising a Unet network model, a first convolutional layer and a second convolutional layer connected in parallel to the output of the Unet network model, wherein the Unet network model comprises an encoder and a decoder, the encoder is used to extract image features of the training sample, and the decoder is used to generate an image feature map based on the image features; the first convolutional layer is used to generate a binary classification prediction result based on the image feature map, and the second convolutional layer is used to generate a ternary classification prediction result based on the image feature map; the space target segmentation network model further comprises an edge network model, the edge perception network model is used to acquire the image features extracted by the encoder, and the edge perception network model is used to acquire a binary edge prediction image and a binary edge loss from the image features; a segmentation result is obtained based on the binary edge prediction image, the binary classification prediction result and the ternary classification prediction result; a symmetry edge loss is determined based on the ternary classification prediction result, a cross-entropy loss is determined based on the binary classification prediction result and the ternary classification prediction result, a loss function is constructed based on the symmetry edge loss, the cross-entropy loss and the binary edge loss, the space target segmentation network model is trained, and a trained space target segmentation network model is obtained; Step S3: acquiring a satellite image containing space targets to be segmented, inputting the satellite image into the trained space target segmentation network model, and obtaining a segmentation result of the satellite image containing space targets to be segmented.

2. The method of claim 1, wherein, The encoder comprises first, second, third, fourth, fifth convolutional blocks and a CBAM module, an SENet module and a decoder connected in sequence, each convolutional block comprises a plurality of convolutional layers and a maximum pooling layer, wherein each convolutional block generates a feature layer, respectively denoted as C1, C2, C3, C4 and C5, and C5 generated by the fifth convolutional block is input into the decoder.

3. The method of claim 2, wherein, The binary classification prediction result comprises a main body and a solar panel part in the space target; and the ternary classification prediction result comprises a main body, a solar panel and an antenna part in the space target.

4. The method of claim 2, wherein, The edge perception network model comprises three parallel branches, each branch comprises a convolutional layer and a channel attention module, the three parallel branches input C2, C3 and C4 which are outputs of the second, third and fourth convolutional modules respectively, the outputs C'2, C'3 and C'4 of the three branches are input into a convolutional layer for splicing the three outputs, the spliced result is mapped to an edge detection result space to obtain a binary edge prediction image, and a binary edge loss is determined based on the binary edge prediction image.

5. The method of claim 4, wherein, The loss function is: Loss = γ * [loss ce (I, L) + loss symmetric (I, α)] + (1 - γ) * loss edge (I edge , L edge ) wherein I is a three-class segmentation prediction result or a two-class segmentation prediction result, I edge is a binary edge prediction image, L edge is a binary edge image label, γ is a weight coefficient, L is a three-class segmentation mask label image or a two-class segmentation mask label image corresponding to I, α is a weight coefficient for calculating the symmetry loss, loss edge (I edge , L edge ) is a binary edge loss, loss symmetric (I, α) is a symmetry edge loss, loss ce (I, L) is a cross-entropy loss of the three-class segmentation prediction result or the two-class segmentation prediction result and the corresponding three-class segmentation mask label image or two-class segmentation mask label image.

6. The method of claim 5, wherein: Wherein, a is a weight coefficient for calculating symmetry loss, flip_img is an image obtained by horizontally or vertically flipping the training sample, and total_pixel is the total number of pixels in I. a = max(similarity_horizon, similarity_vertical) Wherein, similarity_horizon represents a measure of horizontal symmetry, similarity_vertical represents a measure of vertical symmetry, flip_img1 is an image obtained by horizontally flipping the training sample, and flip_img2 is an image obtained by vertically flipping the training sample.

7. The method of claim 6, wherein: loss edge (I edge , L edge ) = CrossEntropy(I edge , L edge , weights) weights = (w neg , w pos ) where w pos represents the weight of the positive sample, w neg represents the weight of the negative sample, neg_num represents the number of negative samples, pos_num represents the number of positive samples, weights represents the sample weight vector, target represents the target edge image, i edge is a binary edge prediction image, and CrossEntropy is a cross-entropy loss function used to measure the difference between the predicted edge image and the real edge information.

8. An image segmentation apparatus characterized by comprising: The device comprises: A training sample set generation module configured to obtain satellite data containing a space target, construct a data set, and determine a training sample set from the data set; A training module configured to input the training sample into a space target segmentation network model, wherein the space target segmentation network model comprises a Unet network model, a first convolutional layer connected in parallel to the output of the Unet network model, and a second convolutional layer, wherein the Unet network model comprises an encoder and a decoder, the encoder is used to extract image features of the training sample, and the decoder is used to generate an image feature map based on the image features; the first convolutional layer generates a binary classification prediction result based on the image feature map, and the second convolutional layer generates a ternary classification prediction result based on the image feature map; the space target segmentation network model further comprises an edge network model, the edge perception network model is used to obtain the image features extracted by the encoder, and the edge perception network model is used to obtain a binary edge prediction image and a binary edge loss from the image features; a segmentation result is obtained based on the binary edge prediction image, the binary classification prediction result, and the ternary classification prediction result; a symmetry edge loss is determined based on the ternary classification prediction result, a cross-entropy loss is determined based on the binary classification prediction result and the ternary classification prediction result, a loss function is constructed based on the symmetry edge loss, the cross-entropy loss, and the binary edge loss, the space target segmentation network model is trained, and a trained space target segmentation network model is obtained; A segmentation module configured to obtain a satellite image containing a space target to be segmented, input the satellite image into the trained space target segmentation network model, and obtain a segmentation result of the satellite image containing the space target to be segmented.

9. An electronic device comprising: a processor configured to execute a plurality of instructions; a memory configured to store the plurality of instructions; wherein the plurality of instructions are stored in the memory and loaded and executed by the processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium having a plurality of instructions stored therein; the plurality of instructions are loaded and executed by a processor to perform the method of any one of claims 1-7.

Images