Star surface multi-class semantic segmentation method and system based on ensemble learning

Abstract

This invention provides a multi-category semantic segmentation method and system for celestial surfaces based on ensemble learning, belonging to the field of target recognition technology. The invention acquires image data of the celestial surface to be segmented; it then processes the acquired celestial surface image data using a pre-trained image segmentation model to obtain semantic segmentation results. The image segmentation model includes a convolutional neural network (CNN) branch module and a deep learning branch module. The CNN branch module extracts lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module extracts global semantic information and long-distance spatial dependencies. A fusion module performs feature-level fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. Finally, a segmentation module further decodes and classifies the integrated feature representation to generate an initial semantic segmentation result.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, specifically to a multi-category semantic segmentation method and system for celestial surfaces based on ensemble learning. Background Technology

[0002] Semantic segmentation is the process of assigning semantic category labels to each pixel in an image. It has developed into a relatively mature technology system in fields such as autonomous driving, Earth remote sensing, and medical imaging. In recent years, deep learning has driven the rapid development of semantic segmentation, with typical approaches falling into two main categories: one is the convolutional neural network approach, represented by DeepLab, which enhances local and multi-scale features through dilated convolution and spatial pyramid pooling; the other is the Transformer approach, represented by SegFormer, which obtains stronger contextual representation capabilities through hierarchical encoders and global modeling mechanisms. However, lunar surface images are not simply substitutes for natural Earth images. Lunar images possess several unique characteristics, such as sparse surface texture features, large shadow spans, highly repetitive terrain structures, and complex grayscale differences. These factors also determine that directly transferring general segmentation networks to lunar surface images often fails to achieve ideal results.

[0003] Based on publicly available information, several studies on lunar vision-related tasks have been conducted in recent years, but the research directions are quite scattered: some works focus on crater detection and segmentation, paying more attention to the identification of single-class impact craters, such as LuSeg, which focuses on the segmentation of positive and negative obstacles; some works focus on the detection of linear terrain such as meandering ravines; some works focus on building lunar simulation datasets to serve navigation and perception, such as LuSNAR, which provides a simulated lunar scene dataset with semantic labels, depth, and point clouds; and some works focus on obstacle segmentation. These studies indicate that a foundation for lunar scene visual understanding exists, but the focus is more on single scenes, single categories, single tasks, or simulated scenarios, and does not equate to solving the problem of multi-class fine segmentation of the complex lunar surface. Furthermore, the lack of an atmosphere on the Moon results in strong light contrast and severe information loss in shadowed areas; the surface structure is mainly composed of craters, rocks, and ruts, with weak texture information and irregular boundaries. These factors make it difficult to achieve ideal results by directly using Earth remote sensing segmentation models for lunar segmentation.

[0004] Most existing single-network segmentation schemes employ general loss functions or simple class weighting, failing to fundamentally address the training bias caused by background pixel dominance and insufficient hard class pixels. This results in models learning average-level optimality rather than optimality for key categories. There is a lack of dedicated reinforcement for hard categories. For key categories such as craters, rocks, tire tracks, and cave entrances, most existing methods lack separate hard-category weights or hard-sample reinforcement modules, leading to occasional hits on key targets but insufficient stability and robustness. Single-structure approaches have limitations. While CNN-type models offer good edge detection, their global semantic understanding is limited; while Transformer-type models have strong global capabilities, they are weak in boundary recovery and small target preservation. Using only one of these methods typically fails to meet the requirements of lunar surface segmentation—capable of clearly seeing the overall terrain relationships while preserving local edge details. The adaptability to multi-scale targets is insufficient. When large craters, small rocks, fine ruts, and mesoscale ridges coexist, single-scale or weakly multi-scale strategies often cannot simultaneously address them, resulting in incomplete outlines of large targets and significant omissions of small targets. Most strategies focus on a single task, emphasizing crater detection, while some focus on gullies, obstacles, or simulated scene navigation, rather than providing a unified solution for fine segmentation of complex lunar surfaces across multiple categories.

[0005] Existing segmentation frameworks using a single semantic segmentation network as their backbone, such as DeepLab-type models based on convolutional neural networks and SegFormer-type models based on Transformers, are directly trained on lunar surface images or applied to output pixel-level classification results. For example, lunar surface segmentation schemes based on DeepLab use CNNs as their backbone and leverage dilated convolutions and multi-scale context to improve segmentation performance. However, their ability to handle global long-range dependencies is relatively limited, and their overall understanding of complex lighting and large-scale structures is insufficient. Existing transfer schemes based on SegFormer or other Transformers utilize hierarchical Transformers to obtain global contextual relationships, enabling a better understanding of scene-level semantics. However, they are generally insufficient in terms of boundary recovery, small object preservation, and specialized enhancements for specific difficult categories. Summary of the Invention

[0006] The purpose of this invention is to provide a multi-class semantic segmentation method and system for celestial surfaces based on ensemble learning. This method addresses the problems of insufficient adaptability to complex lunar scenes, weak recognition of difficult categories, and poor robustness of segmentation results in existing solutions. An ensemble segmentation method with both global semantic modeling and local detail representation capabilities is proposed, while simultaneously improving the model's segmentation performance under class imbalance and complex lighting variations. By constructing a dual-branch structure of convolutional neural network branches and Transformer branches, feature-level ensemble of heterogeneous models is achieved, enabling the model to simultaneously capture local texture information and global semantic relationships. By introducing weighted cross-entropy loss, Dice loss, and Lovasz loss into the total loss function, collaborative constraints on class imbalance and regional overlap are implemented, thereby improving the model's ability to recognize low-proportion difficult categories. A reinforcement training strategy oriented towards difficult categories is introduced. Through class weight adjustment, priority sampling of difficult samples, and local region selection, the excessive bias of the model towards high-frequency categories during training is alleviated, improving the model's learning ability for key terrain categories such as craters, rocks, and cave entrances. Through a multi-level ensemble learning framework, the generated semantic segmentation model has higher accuracy and better adaptability to practical applications, making it more suitable for real-world application scenarios such as lunar surface path planning and environmental understanding.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] In a first aspect, the present invention provides a multi-class semantic segmentation method for celestial surfaces based on ensemble learning, comprising:

[0009] Acquire image data of the surface of the star to be segmented;

[0010] A pre-trained image segmentation model is used to process the acquired celestial surface image data to obtain semantic segmentation results. The image segmentation model includes a convolutional neural network (CNN) branch module and a deep learning branch module. The CNN branch module extracts lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module extracts global semantic information and long-range spatial dependencies. A fusion module performs feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. A segmentation module further decodes and classifies the integrated feature representation to generate an initial semantic segmentation result.

[0011] As a further limitation of the first aspect of the present invention, the image segmentation model is trained by a training set, which includes multiple images of the surface of celestial bodies and labels indicating the categories of entities in the images.

[0012] As a further limitation of the first aspect of the present invention, during the training phase, a composite loss function consisting of weighted cross-entropy loss, Dice loss, and Lovasz loss is used to achieve optimization-level ensemble; weighted cross-entropy is used to alleviate class imbalance, increasing the contribution of difficult classes to the loss; Dice Loss is used to optimize prediction from the perspective of region overlap, improving the segmentation completeness of small and irregular targets; Lovasz loss is used to more directly optimize IoU performance, making the training objective more consistent with the evaluation objective.

[0013] As a further limitation of the first aspect of the present invention, a difficult category reinforcement training strategy is introduced to enhance the weight of key categories with low proportions, improve the model's sensitivity to difficult category features, and prevent excessive bias towards background regions.

[0014] As a further limitation of the first aspect of the present invention, the fusion module consists of two parallel branches: one branch is DeepLabV3-ResNet50, and the other branch is SegFormer-B0; the same input image is fed into these two branches simultaneously, DeepLabV3 outputs a segmentation logits map, and SegFormer also outputs a segmentation logits map, and the two are weighted and summed according to a fixed ratio, so that the model achieves a balance between clear edges and overall semantic integrity.

[0015] As a further limitation of the first aspect of the present invention, multiple images of the surface of stars and simulated star scenes are acquired. After image size normalization, random cropping, random flipping and local patch extraction, the data preprocessing is performed. The patch selection of training samples is combined with the proportion of difficult categories to perform biased sampling, so that the model can come into contact with more regions containing key categories during training.

[0016] Secondly, the present invention provides a multi-class semantic segmentation system for celestial surfaces based on ensemble learning, comprising:

[0017] The acquisition module is used to acquire image data of the surface of the star to be segmented;

[0018] The processing module is used to process the acquired celestial surface image data using a pre-trained image segmentation model to obtain semantic segmentation results of the celestial surface image. The image segmentation model includes a convolutional neural network branch module and a deep learning branch module. The convolutional neural network branch module is used to extract lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module is used to extract global semantic information and long-distance spatial dependencies. The fusion module is used to perform feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. The segmentation module is used to further decode and classify the integrated feature representation to generate an initial semantic segmentation result.

[0019] Thirdly, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the multi-category semantic segmentation method for celestial surfaces based on ensemble learning as described in the first aspect.

[0020] Fourthly, the present invention provides a computer device including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the multi-class semantic segmentation method for star surfaces based on ensemble learning as described in the first aspect.

[0021] Fifthly, the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the multi-class semantic segmentation method for star surfaces based on ensemble learning as described in the first aspect.

[0022] Terminology Explanation:

[0023] Semantic segmentation of celestial (lunar) surfaces: This refers to classifying each pixel in an image of a celestial (lunar) surface into a predefined semantic category, thus assigning different terrain regions to predefined semantic categories. The semantic categories involved in this invention include plains, craters, rocks, gullies, tire tracks, mountains, and cave entrances. Semantic category: A predefined terrain category label in the segmentation task. In this invention, there are seven semantic categories: plain, crater, rock, gully, track, mountain, and cave. Hard category: A category in the training data that has a small area, sparse distribution, complex boundaries, and is prone to being missed or misclassified. In this invention, crater, rock, and track are mainly considered hard categories. Dual-branch fusion model: A model structure that processes the same input image through two different feature extraction branches, and then fuses the outputs of the two branches. In this invention, one branch focuses on local feature structure extraction, and the other branch focuses on global semantic feature extraction. Logits graph: The original classification score graph output by the model at each pixel location for each semantic category. The logits graph is not the final segmentation result, but rather an intermediate representation generated after subsequent class selection. Weighted cross-entropy loss: Based on the ordinary cross-entropy loss, different weights are assigned to different classes, making the model more sensitive to classification errors in difficult classes during training.

[0024] Beneficial effects of this invention:

[0025] (1) This invention simultaneously considers global semantic information and local detail features. While existing segmentation methods based on convolutional neural networks have certain advantages in edge detail and local texture extraction, their global modeling capabilities are limited, making it difficult to effectively understand long-distance spatial relationships in complex lunar scenes. On the other hand, while segmentation methods based on Transformers have strong global semantic modeling capabilities, they are insufficient in preserving small targets and boundary details. This invention achieves joint modeling of local detail information and global semantic information by constructing a heterogeneous fusion structure of CNN and Transformer dual branches.

[0026] (2) This invention significantly improves the ability to identify difficult categories. Existing general segmentation methods usually focus more on the overall average accuracy, and are easily dominated by high-frequency background categories under class imbalance conditions, resulting in insufficient identification ability of low-proportion key categories such as craters, rocks, and tire tracks. This invention proposes a difficult category reinforcement training mechanism, which can significantly improve the recall rate of difficult categories and the segmentation ability of small objects.

[0027] (3) This invention is more suitable for class imbalance scenarios. Most existing methods use a single loss function for optimization, which makes it difficult to simultaneously take into account class balance and region integrity, resulting in class bias problems in complex scenarios. This invention constructs a composite loss function composed of weighted cross-entropy loss, Dice loss and Lovasz class loss, which realizes multi-objective collaborative optimization, improves the gradient contribution of difficult classes in model training, and thus improves the overall segmentation performance.

[0028] (4) This invention can improve robustness and generalization ability in complex environments. Due to the complexity of the lunar environment, traditional segmentation models are prone to prediction instability under different environmental conditions. This invention further adopts a decision-level fusion mechanism based on ensemble learning, which can effectively reduce the impact of illumination changes and input scale changes compared with existing technologies.

[0029] (5) This invention forms a complete multi-level integrated learning framework. Existing technologies usually only improve the network structure or a single optimization module, lacking a systematic overall optimization design. This invention realizes joint modeling of local and global information, multi-objective collaborative optimization, and fusion of multiple prediction results and robustness enhancement. Compared with traditional single-module optimization schemes, it has stronger overall collaborative capabilities and engineering application value.

[0030] The advantages of additional aspects of the invention will be set forth more clearly in the following description or will be learned by practice of the invention. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a flowchart of the multi-category semantic segmentation method for celestial surfaces based on ensemble learning, as described in an embodiment of the present invention.

[0033] Figure 2 This is a block diagram illustrating the feature fusion principle described in an embodiment of the present invention.

[0034] Figure 3 This is a diagram showing the lunar surface segmentation results according to an embodiment of the present invention. The diagram is divided into four columns from top to bottom, each column representing: plains, craters, and wheel tracks; plains, craters, rock blocks, and wheel tracks; plains, craters, rock blocks, and ravines; and plains, rock blocks, and cave entrances.

[0035] Figure 4This is a diagram showing the lunar surface segmentation results according to an embodiment of the present invention. The diagram is divided into four columns from top to bottom, each column containing: plains, craters, rock blocks, and mountains; plains, craters, rock blocks, and ravines; plains and craters; plains, craters, and rock blocks. Detailed Implementation

[0036] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0037] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0038] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.

[0039] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0040] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0041] To facilitate understanding of the present invention, the present invention will be further explained and described below with reference to the accompanying drawings and specific embodiments, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0042] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing the present invention.

[0043] This invention is a multi-class semantic segmentation method for complex scenes on the surface of celestial bodies (especially the moon), and aims to solve the following technical problems:

[0044] The inherent imaging challenges of celestial (especially lunar) scenes lead to unstable segmentation features. Because celestial bodies (especially the moon) lack atmospheric scattering buffers, the contrast in illumination is often extremely high, with significant differences between bright and shadow areas. Furthermore, the surface texture of celestial bodies (especially the moon) is relatively monotonous, with small texture differences between many categories. Therefore, models struggle to stably distinguish targets based solely on local texture. This is particularly true in areas like ruts, gullies, low-lying rocks, and meteorite regions with indistinct edges, where misclassification and missed classification are common. The severely imbalanced category distribution leads to training bias issues. In celestial (especially lunar) images, large, flat background areas typically dominate, while key targets such as craters, rocks, and ruts have a much lower pixel percentage than the background. Conventional segmentation models, when trained using uniform loss and uniform sampling, tend to prioritize learning categories with high percentages and easy recognition, resulting in insufficient learning of small-sample and difficult categories. A single-structure network struggles to simultaneously consider both global semantics and local details. CNN-based methods excel at extracting edges and local textures, but have limited global dependency modeling capabilities. Transformer segmentation methods, on the other hand, are adept at capturing long-range dependencies and overall scene semantics, but often fall short in preserving details of small targets. The DeepLab architecture enhances local to mesoscale representations through dilated convolutions and multi-scale context, while the SegFormer architecture achieves efficient global semantic modeling through hierarchical Transformer encoders and lightweight decoders. Both approaches have their advantages, but neither alone can fully meet the demands of complex lunar surface segmentation. Therefore, this invention provides a multi-class semantic segmentation method based on ensemble learning, applicable to complex surface scenes of celestial bodies (especially the Moon). Through heterogeneous model integration, collaborative optimization of composite losses, and decision optimization during the inference phase, the model structurally balances global semantics and local details. Furthermore, it enhances the segmentation capability and robustness of multi-scale targets during training and inference, thereby improving the model's overall segmentation performance and key category recognition capabilities for complex lunar surface scenes.

[0045] Example 1

[0046] In this embodiment 1, a multi-category semantic segmentation system for celestial surfaces based on ensemble learning is first provided, comprising: an acquisition module for acquiring image data of the celestial surface to be segmented; a processing module for processing the acquired celestial surface image data using a pre-trained image segmentation model to obtain semantic segmentation results of the celestial surface image; wherein the image segmentation model includes a convolutional neural network branch module and a deep learning branch module, wherein the convolutional neural network branch module is used to extract lunar surface texture, edges, and fine-grained structural features, and the deep learning branch module is used to extract global semantic information and long-distance spatial dependencies; a fusion module for performing feature-level fusion of the features output by the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation; and a segmentation module for further decoding and classification of the integrated feature representation to generate an initial semantic segmentation result.

[0047] In this embodiment, the above-described system is used to implement a multi-category semantic segmentation method for celestial surfaces based on ensemble learning. The method includes: first, acquiring image data of the celestial surface to be segmented using an acquisition module; then, using a processing module with a pre-trained image segmentation model to process the acquired celestial surface image data to obtain semantic segmentation results; wherein the image segmentation model includes a convolutional neural network branch module and a deep learning branch module, where the convolutional neural network branch module is used to extract lunar surface texture, edges, and fine-grained structural features, and the deep learning branch module is used to extract global semantic information and long-distance spatial dependencies; a fusion module is used to perform feature-level fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation; and a segmentation module is used to further decode and classify the integrated feature representation to generate an initial semantic segmentation result.

[0048] The image segmentation model is trained using a training set, which includes multiple images of celestial surfaces and labels indicating entity categories within the images. During training, a composite loss function consisting of weighted cross-entropy loss, Dice loss, and Lovasz loss is employed to achieve optimized ensemble training. Weighted cross-entropy is used to mitigate class imbalance, increasing the contribution of difficult categories to the loss. Dice loss optimizes predictions from the perspective of region overlap, improving the segmentation completeness of small and irregular targets. Lovasz loss directly optimizes IoU performance, aligning the training and evaluation objectives more closely. A hard-category reinforcement training strategy is introduced, increasing the weights of low-proportion key categories to enhance the model's sensitivity to hard-category features and prevent excessive bias towards background regions.

[0049] The fusion module consists of two parallel branches: one is DeepLabV3-ResNet50, and the other is SegFormer-B0. The same input image is fed into these two branches simultaneously. DeepLabV3 outputs a segmentation logits map, and SegFormer also outputs a segmentation logits map. The two are weighted and summed according to a fixed ratio to achieve a balance between edge clarity and overall semantic integrity in the model.

[0050] In acquiring the training set, multiple images of celestial surfaces and simulated celestial scenes were obtained. After image size normalization, random cropping, random flipping, and local patch extraction, the data preprocessing was performed. The selection of training samples patches was combined with the proportion of difficult categories to perform biased sampling, so that the model could come into contact with more regions containing key categories during training.

[0051] Example 2

[0052] In this embodiment, taking lunar surface image data segmentation as an example, a multi-category semantic segmentation method for lunar surface based on ensemble learning is provided. The overall flowchart of the method is as follows: Figure 1 As shown.

[0053] First, lunar surface image data is acquired and preprocessed, including cropping, normalization, data augmentation, and label format conversion, to generate images and pixel-level labeled data suitable for model training. Then, the preprocessed lunar images are input into a two-branch semantic segmentation network based on ensemble learning. The network includes a convolutional neural network branch and a Transformer branch. The convolutional neural network branch extracts lunar surface texture, edges, and fine-grained structural features, while the Transformer branch extracts global semantic information and long-range spatial dependencies. Next, the features output from the two branches are fused at the feature level, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. The fused features are further output to a decoding and classification module to generate initial semantic segmentation results. During the training phase, a composite loss function consisting of weighted cross-entropy loss, Dice loss, and Lovasz loss is used to achieve optimized ensemble. Simultaneously, a hard category reinforcement training strategy is introduced, with weights augmented for low-proportion key categories such as craters, rocks, and ruts to improve the model's ability to recognize hard categories. During the inference phase, this invention further employs a decision-level integration strategy to predict the multi-scale, flipping, and other enhancement results of the same input image, and fuses multiple prediction results to finally output a stable multi-class semantic segmentation map of the lunar surface.

[0054] The data preprocessing module takes input images from real lunar images and simulated lunar scene images. After image size normalization, random cropping, random flipping, and local patch extraction, the data preprocessing allows for biased sampling of training samples based on the proportion of difficult categories, enabling the model to access more regions containing key categories during training.

[0055] The convolutional branch is primarily responsible for extracting edge texture and local geometric details. The method employs a DeepLabV3 architecture with ResNet50 as its backbone, utilizing dilated convolutions to expand the receptive field and capturing context at different scales through a multi-scale dilation module. The DeepLab model segmentation method achieves a good balance between boundary and context modeling.

[0056] The Transformer branch is primarily responsible for learning scene-level global relationships and long-distance dependencies. It employs the SegFormer-B0 structure, whose hierarchical encoder generates multi-scale semantic features without relying on positional encoding interpolation. Its lightweight decoding design also contributes to improved efficiency. The Transformer model segmentation method can comprehensively understand the extension directions of mountains, craters, and gullies, as well as the relationships between multiple regions.

[0057] Since the convolutional and Transformer branches encode different types of information, it is difficult to simultaneously capture both local fine details and global context if only one is used. Therefore, this embodiment uses a feature fusion module to jointly model the information from both branches. In this method, the two branches are fused using fixed weights.

[0058] Network principle block diagram as follows Figure 2 As shown, the fusion model consists of two parallel branches: one is DeepLabV3-ResNet50, and the other is SegFormer-B0. The same input image is fed into both branches simultaneously. DeepLabV3 outputs a segmentation logits map, and SegFormer also outputs a segmentation logits map. The two are then weighted and summed according to a fixed ratio.

[0059] ;

[0060] in, This is the logits output by the DeepLabV3 branch. It is the logits output by the SegFormer branch. This is the fusion coefficient. After training, it was found that when... The model training performance is optimal at this time, therefore fixed fusion weights are set: This means that each branch has half the weight. The feature fusion module allows the model to achieve a balance between clear edges and overall semantic integrity.

[0061] This embodiment employs a composite loss function consisting of three parts to address the issues of class imbalance and insufficient recognition of difficult classes in lunar surface semantic segmentation tasks. Compared to traditional single loss functions, this multi-objective joint constraint approach improves overall segmentation performance and key class recognition capabilities. Weighted cross-entropy is used to alleviate class imbalance, increasing the contribution of difficult classes to the loss; Dice Loss optimizes prediction from the perspective of region overlap, improving the segmentation completeness of small and irregular targets; and Lovasz-type loss directly optimizes IoU performance, making the training and evaluation objectives more consistent. Through loss coupling, pixel-level classification, region-level overlap, and evaluation metrics can be considered simultaneously.

[0062] Therefore, the formula for calculating the total loss is as follows:

[0063]

[0064] in, This represents the weighted cross-entropy loss; This indicates Dice's loss. Represents the Lovasz class loss. These are the weighting coefficients for the corresponding loss terms.

[0065] Hard category reinforcement is a key differentiator from other general two-branch segmentation schemes. Higher category weights are assigned to hard categories to increase their contribution to the loss function. Specifically, categories such as craters, rocks, and tire tracks are given higher loss weights than background categories, enhancing the model's sensitivity to hard category features, effectively mitigating class imbalance, and preventing the model from overly favoring background regions.

[0066] During model training, the weights were set as follows: plain=1.0, crater=1.65, rock=1.65, gully=1.2, track=1.35, mountain=1.1, and cave=1.1. This means that the crater, rock, and track categories were assigned larger loss weights. This means that the loss from misclassification of these categories is greater, and the model will pay more attention to these categories during training, thus mitigating the impact of class imbalance.

[0067] Table 1 Training results of the lunar segmentation model

[0068]

[0069] The training results of the lunar multi-class semantic segmentation model are shown in Table 1. Experiments were conducted using a single DeepLabV3 model, a single SegFormer model, a fusion model of DeepLabV3 and SegFormer, and a fusion model with larger class weights for the more difficult-to-segment categories such as crater, rock, and track. Overall, Experiment 4 achieved the best performance to date, reaching an accuracy of 95.51%, mIoU of 90.45%, and Dice of 94.98% on the test set, thus initially achieving the goal of over 90% accuracy for the lunar segmentation model.

[0070] In the optimal model shown in Experiment 4, the segmentation results for each category are shown in Table 2.

[0071] Table 2. Segmentation results for each category in the model.

[0072]

[0073] Based on the classification results, the plain and cave classes showed good segmentation performance, with IoU exceeding 92%; the crater, gully, and track classes all achieved around 90%, demonstrating relatively stable overall performance. The rock class remains the most challenging class, indicating that there is still room for further optimization for fragmented targets and small-scale regions.

[0074] The model segmentation results are as follows Figure 3 , 4 As shown, gray represents plains, red represents craters, blue represents rocks, purple represents gullies, yellow represents tracks, green represents mountains, and blue-green represents cave entrances. Each row contains four images from left to right: the original image, the manually labeled ground plane (GT) image, the prediction result image, and the overlay visualization.

[0075] Example 3

[0076] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they implement the multi-category semantic segmentation method for celestial surfaces based on ensemble learning as described above. The method includes:

[0077] Acquire image data of the surface of the star to be segmented;

[0078] A pre-trained image segmentation model is used to process the acquired celestial surface image data to obtain semantic segmentation results. The image segmentation model includes a convolutional neural network (CNN) branch module and a deep learning branch module. The CNN branch module extracts lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module extracts global semantic information and long-range spatial dependencies. A fusion module performs feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. A segmentation module further decodes and classifies the integrated feature representation to generate an initial semantic segmentation result.

[0079] Example 4

[0080] This embodiment 4 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, and the memory stores program instructions executable by the processor. The processor calls the program instructions to execute the multi-class semantic segmentation method for celestial surfaces based on ensemble learning as described above, the method including:

[0081] Acquire image data of the surface of the star to be segmented;

[0082] A pre-trained image segmentation model is used to process the acquired celestial surface image data to obtain semantic segmentation results. The image segmentation model includes a convolutional neural network (CNN) branch module and a deep learning branch module. The CNN branch module extracts lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module extracts global semantic information and long-range spatial dependencies. A fusion module performs feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. A segmentation module further decodes and classifies the integrated feature representation to generate an initial semantic segmentation result.

[0083] Example 5

[0084] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein, the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the multi-class semantic segmentation method for celestial surfaces based on ensemble learning as described above. The method includes:

[0085] Acquire image data of the surface of the star to be segmented;

[0086] A pre-trained image segmentation model is used to process the acquired celestial surface image data to obtain semantic segmentation results. The image segmentation model includes a convolutional neural network (CNN) branch module and a deep learning branch module. The CNN branch module extracts lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module extracts global semantic information and long-range spatial dependencies. A fusion module performs feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. A segmentation module further decodes and classifies the integrated feature representation to generate an initial semantic segmentation result.

[0087] In summary, the multi-class semantic segmentation method for celestial surfaces based on ensemble learning described in this invention proposes a dual-branch heterogeneous feature extraction framework based on convolutional neural networks (CNN) and Transformer structures. By constructing two branches with different feature expression capabilities, it achieves collaborative modeling of local detailed features and global semantic information. A reinforcement training mechanism is proposed for the low-proportion difficult categories on the lunar surface to address the insufficient ability of existing models to identify key categories. A composite loss optimization mechanism suitable for complex lunar scenarios is proposed, achieving synergistic optimization of category balance, regional integrity, and evaluation index consistency by combining multiple optimization objectives. Weighted cross-entropy loss, Dice loss, and Lovasz-class loss are used to construct the loss function. A holistic semantic segmentation framework combining structural design, training optimization, and inference enhancement is proposed. This invention constructs a complete multi-level ensemble learning framework from the perspectives of feature extraction, feature fusion, loss optimization, and difficult category training.

[0088] The core idea of ​​this invention is to improve the performance of multi-class semantic segmentation in complex lunar surface scenes by using a multi-level semantic segmentation framework based on ensemble learning, combined with heterogeneous feature extraction, hard category reinforcement training, and collaborative optimization of composite losses. Therefore, any structural design, training mechanism, or optimization method with equivalent or similar technical ideas can achieve the purpose of this invention.

[0089] In this invention, the convolutional neural network branch preferably adopts the DeepLabV3-ResNet50 structure for extracting local texture, edge, and fine-grained features. However, in practical applications, other convolutional neural network structures with local feature extraction capabilities can be used as alternatives, such as U-Net and ResNet101, which can also achieve local texture and boundary feature extraction and can be used as equivalent alternatives to the convolutional branch of this invention.

[0090] In this invention, the Transformer branch preferably adopts the SegFormer structure for modeling global semantic information and long-distance spatial dependencies. However, this invention is not limited to this specific structure, and other Transformer-type visual encoding structures, such as Vision Transformer and BEiT, can also be used. These Transformer-type structures can all achieve global context information modeling and are therefore alternative solutions to the Transformer branch of this invention.

[0091] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0093] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0094] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment, whereby a series of operational steps are performed to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0095] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that, based on the technical solutions disclosed in the present invention, various modifications or variations that can be made by those skilled in the art without creative effort should be included within the scope of protection of the present invention.

Claims

1. A multi-class semantic segmentation method for celestial surfaces based on ensemble learning, characterized in that, include: Acquire image data of the surface of the star to be segmented; A pre-trained image segmentation model is used to process the acquired celestial surface image data to obtain semantic segmentation results. The image segmentation model includes a convolutional neural network (CNN) branch module and a deep learning branch module. The CNN branch module extracts lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module extracts global semantic information and long-range spatial dependencies. A fusion module performs feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. A segmentation module further decodes and classifies the integrated feature representation to generate an initial semantic segmentation result.

2. The multi-class semantic segmentation method for celestial surfaces based on ensemble learning according to claim 1, characterized in that, The image segmentation model is trained using a training set, which includes multiple images of the surface of celestial bodies and labels indicating the categories of entities in the images.

3. The multi-class semantic segmentation method for celestial surfaces based on ensemble learning according to claim 1, characterized in that, During the training phase, a composite loss function consisting of weighted cross-entropy loss, Dice loss, and Lovasz loss is used to achieve optimization-level ensemble. Weighted cross-entropy is used to alleviate class imbalance, increasing the contribution of difficult classes to the loss. Dice Loss is used to optimize prediction from the perspective of region overlap, improving the segmentation completeness of small and irregular targets. Lovasz loss is used to more directly optimize IoU performance, making the training objective more consistent with the evaluation objective.

4. The multi-class semantic segmentation method for celestial surfaces based on ensemble learning according to claim 3, characterized in that, A training strategy to enhance the difficulty category is introduced, which strengthens the weights of key categories with low proportions, improves the model's sensitivity to difficult category features, and prevents excessive bias towards background regions.

5. The multi-class semantic segmentation method for celestial surfaces based on ensemble learning according to claim 1, characterized in that, The fusion module consists of two parallel branches: one branch is DeepLabV3-ResNet50, and the other branch is SegFormer-B0. The same input image is fed into these two branches simultaneously. DeepLabV3 outputs a segmentation logits map, and SegFormer also outputs a segmentation logits map. The two are weighted and summed according to a fixed ratio to achieve a balance between edge clarity and overall semantic integrity in the model.

6. The multi-class semantic segmentation method for celestial surfaces based on ensemble learning according to claim 2, characterized in that, Multiple images of celestial surfaces and simulated celestial scenes were acquired. After image size normalization, random cropping, random flipping, and local patch extraction, the data preprocessing was performed. The selection of training samples for patches was combined with the proportion of difficult categories to perform biased sampling, so that the model could come into contact with more regions containing key categories during training.

7. A multi-class semantic segmentation system for celestial surfaces based on ensemble learning, characterized in that, include: The acquisition module is used to acquire image data of the surface of the star to be segmented; The processing module is used to process the acquired celestial surface image data using a pre-trained image segmentation model to obtain semantic segmentation results of the celestial surface image. The image segmentation model includes a convolutional neural network branch module and a deep learning branch module. The convolutional neural network branch module is used to extract lunar surface texture, edges, and fine-grained structural features, while the deep learning branch module is used to extract global semantic information and long-distance spatial dependencies. The fusion module is used to perform feature-level integration and fusion of the features output from the two branches, jointly modeling local detail features and global semantic features to obtain a fused integrated feature representation. The segmentation module is used to further decode and classify the integrated feature representation to generate an initial semantic segmentation result.

8. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium is used to store computer instructions, which, when executed by a processor, implement the multi-class semantic segmentation method for celestial surfaces based on ensemble learning as described in any one of claims 1-6.

9. A computer device, characterized in that, The system includes a memory and a processor, the processor and the memory communicating with each other, the memory storing program instructions executable by the processor, and the processor calling the program instructions to execute the multi-class semantic segmentation method for star surfaces based on ensemble learning as described in any one of claims 1-6.

10. An electronic device, characterized in that, include: The electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the multi-class semantic segmentation method for celestial surfaces based on ensemble learning as described in any one of claims 1-6.

Images